Multicoreexpo2011 05-05fridiusingtemplate-110518193827-phpapp02
-
Upload
myra-jeannette-fitzgerald -
Category
Technology
-
view
146 -
download
0
Transcript of Multicoreexpo2011 05-05fridiusingtemplate-110518193827-phpapp02
1
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
aicas Technology
Multicore for Real-Timeand Safety-Critical Software:
Avoid the Pitfal ls
Dr. Fridtjof Siebert, CTO, aicasDr. James J. Hunt, CEO aicas
MultiCoreExpo 2011, 5th May 2011
2
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Agenda
Race conditions
Synchronization
Atomic operations
Memory model
Values out of thin air
Reaching peak performance
CPU affinities
Multicore scheduling
Lock free algorithms
Compare and Swap
3
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Typical Problems on Mult icore
typical code sequence (C/C++ or Java)
int counter;
void increment(){ counter++; }
4
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Typical Problems on Mult icore
typical code sequence (C/C++ or Java)
int counter;
void increment(){ counter++; } r1 = counter;
r2 = r1 + 1; counter = r2;
5
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Typical Problems on Mult icore
typical code sequence (C/C++ or Java)
int counter;
void increment(){ counter++; } r1 = counter;
r2 = r1 + 1; counter = r2;
r1 = counter; r2 = r1 + 1; counter = r2;
Thread 1 Thread 2
6
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Typical Problems on Mult icore
typical code sequence (C/C++ or Java)
int counter;
void increment(){ counter++; } r1 = counter;
r2 = r1 + 1; counter = r2;
r1 = counter; r2 = r1 + 1; counter = r2;
Thread 1 Thread 2
An increment can get lost!
7
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Typical Problems on Mult icore
typical code sequence (C/C++ or Java)
int counter;
void increment(){ counter++; }
8
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Typical Problems on Mult icore
typical code sequence (C/C++ or Java)
int counter;
void increment(){ counter++; }
code lacks synchronization
but on a single core, it practically always works!
on a multicore, chances for failure explode!
9
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Synchronization
solution: synchronize
int counter;
synchronized void increment(){ counter++; }
Easy, problem solved.
Right? See later.
10
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Atomic Operations
What is the result of
int a, b; /* 32 bit, initially 0 */
Thread 1 Thread 2 b = a; a = 1;
?
11
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Atomic Operations
What is the result of
int a, b; /* 32 bit, initially 0 */
Thread 1 Thread 2 b = a; a = 1;
?
b == 0b == 1
12
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Atomic Operations
What is the result of
long a, b; /* 64 bit, initially 0 */
Thread 1 Thread 2 b = a; a = 1;
?
13
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Atomic Operations
What is the result of
long a, b; /* 64 bit, initially 0 */
Thread 1 Thread 2 b = a; a = 1;
?
b == 0b == 1b == 4294967296b == 4294967295
14
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Cache Structure
CPUs may have local caches for performance
CPU0 CPU1CPU0 CPU1CPU0 CPU1CPU0 CPU1 CPU0 CPU1CPU0 CPU1CPU0 CPU1CPU2 CPU3
CPU0 CPU1CPU0 CPU1L1 Cache L1 CacheL1 Cache L1 Cache
L2 Cache L2 Cache
Main Memory
15
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Cache Structure
Modifications do not become visible immediately
Modifications may be re-ordered
Reads may refer to outdated (cached) data
Reads may be re-ordered
CPU0 CPU1 CPU2 CPU3
L1 Cache L1 CacheL1 Cache L1 Cache
L2 Cache L2 Cache
Main Memory
16
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Typical Problems on Mult icore
polling update
long counter;[..]do { doSomething(); }while (counter < MAX);
counter is incremented by parallel thread
on a Multicore, changes to counter may not become visible!
17
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Typical Problems on Mult icore
polling update
long counter;[..]do { doSomething(); }while (counter < MAX);
counter is incremented by parallel thread
on a Multicore, changes to counter may not become visible!
18
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Solution: volati le?
polling update
volatile long counter;[..]do { doSomething(); }while (counter < MAX);
works for Java
does not work for C!
19
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Solution: volati le?
polling update
volatile long counter;[..]do { doSomething(); }while (counter < MAX);
works for Java
does not work for C!
20
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Understanding the Memory Model
Memory model specifies what optimizations are permitted by the compiler or underlying hardware
C/C++ programs have undefined semantics in case of race conditions
Java defines a strict memory model
21
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Java's Memory Model
ordering operations aresynchronized block
accessing a volatile variable
The presence of an ordering operation determines the visible state in shared memory
22
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Java's Memory Model: Defined Order
all reads are completed before entering synchronized block, or
reading a volatile variable
read fence
all writes are completed before exiting a synchronized block, or
writing a volatile variable
write fence
23
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Java's Memory Model: Data Races
data races are not forbidden in Java you can use shared memory variables
your code has to tolerate optimizations
examples collecting debugging / profiling information
useful if occasional errors due to data races are tolerable
24
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Memory Model Example
Shared memory communicationPtr p;boolean p_valid;
Thread 1
p = new Ptr(); p_valid = true;
25
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Memory Model Example
Shared memory communicationPtr p;boolean p_valid;
Thread 1 Thread 2
p = new Ptr(); p_valid = true;
26
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Memory Model Example
Shared memory communicationPtr p;boolean p_valid;
Thread 1 Thread 2
p = new Ptr(); if (p_valid)p_valid = true; p.call();
27
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Memory Model Example
Shared memory communication
Thread 1 Thread 2 p = new Ptr(); if (p_valid)p_valid = true; p.call();
28
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Memory Model Example
Shared memory communication
Thread 1 Thread 2 p = new Ptr(); if (p_valid)p_valid = true; p.call();
What may happen:
29
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Memory Model Example
Shared memory communication
Thread 1 Thread 2 p = new Ptr(); if (p_valid)p_valid = true; p.call();
What may happen: t1 = new Ptr(); t2 = true; p_valid = t2; p = t1;
30
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Memory Model Example
Shared memory communication
Thread 1 Thread 2 p = new Ptr(); if (p_valid)p_valid = true; p.call();
What may happen: t1 = new Ptr(); t2 = true; p_valid = t2; p = t1;
Writes reordered!
31
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Memory Model Example
Shared memory communication
Thread 1 Thread 2 p = new Ptr(); if (p_valid)p_valid = true; p.call();
What may happen: t1 = new Ptr(); t2 = true; p_valid = t2; p = t1;
Writes reordered!
32
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Memory Model Example
Shared memory communication
Thread 1 Thread 2 p = new Ptr(); if (p_valid)p_valid = true; p.call();
What may happen: t1 = new Ptr(); t3 = p;t2 = true; if (p_valid) p_valid = t2; t3.call(); p = t1;
Writes reordered! Reads reordered!
33
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Memory Model Example
Shared memory communication
Thread 1 Thread 2 p = new Ptr(); if (p_valid)p_valid = true; p.call();
What may happen: t1 = new Ptr(); t3 = p;t2 = true; if (p_valid) p_valid = t2; t3.call(); p = t1;
Writes reordered! Reads reordered!
34
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Shared memory communicationvolatile Ptr p;volatile boolean p_valid;
Thread 1 Thread 2 p = new Ptr(); if (p_valid)p_valid = true; p.call();
in Java
Example Use of Java's Memory Model
35
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Shared memory communicationvolatile Ptr p;volatile boolean p_valid;
Thread 1 Thread 2 p = new Ptr(); if (p_valid)p_valid = true; p.call();
in Java
Example Use of Java's Memory Model
36
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Shared memory communicationvolatile Ptr p;volatile boolean p_valid;
Thread 1 Thread 2 p = new Ptr(); if (p_valid)p_valid = true; p.call();
in Java
Example Use of Java's Memory Model
37
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Shared memory communicationvolatile Obj *p;volatile boolean p_valid;
Thread 1 Thread 2 p = malloc(..); if (p_valid)p_valid = TRUE; p>f = ..;();
in C?
Example Use of C's Memory Model
38
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Shared memory communicationvolatile Obj *p;volatile boolean p_valid;
Thread 1 Thread 2 p = malloc(..); if (p_valid)p_valid = TRUE; p>f = ..;();
in C? CPU may reorder memory accesses!
Example Use of C's Memory Model
39
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Shared memory communicationvolatile Obj *p;volatile boolean p_valid;
Thread 1 Thread 2 p = malloc(..); if (p_valid)p_valid = TRUE; p>f = ..;();
in C? CPU may reorder memory accesses!
Example Use of C's Memory Model
40
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Shared memory communicationvolatile Obj *p;volatile boolean p_valid;
Thread 1 Thread 2 p = malloc(..); if (p_valid)p_valid = TRUE; p>f = ..;();
in C? CPU may reorder memory accesses!
Example Use of C's Memory Model
41
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Shared memory communicationvolatile Obj *p;volatile boolean p_valid;
Thread 1 Thread 2 p = malloc(..); if (p_valid)p_valid = TRUE; p>f = ..;
How to fix it?
Example Use of C's Memory Model
42
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Shared memory communicationvolatile Obj *p;volatile boolean p_valid;
Thread 1 Thread 2 p = malloc(..); if (p_valid)asm volatile( p>f = ..; "sfence":::"memory"); p_valid = TRUE;
How to fix it? Add memory fences!
Example Use of C's Memory Model
43
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Shared memory communicationvolatile Obj *p;volatile boolean p_valid;
Thread 1 Thread 2 p = malloc(..); if (p_valid)asm volatile( { "sfence":::"memory"); asm volatile(p_valid = TRUE; "lfence":::"memory"); p>f = ..; }
How to fix it? Add memory fences!
Example Use of C's Memory Model
44
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Example Use of C's Memory Model
Shared memory communicationvolatile Obj *p;volatile boolean p_valid;
Thread 1 Thread 2 p = malloc(..); if (p_valid)asm volatile( { "sfence":::"memory"); asm volatile(p_valid = TRUE; "lfence":::"memory"); p>f = ..; }
How to fix it? Add memory fences!
45
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Out of Thin Air
imagine this codeint x = 0, n = 0;
Thread 1 Thread 2 for (i=0; i<n; i++) x = 42; x += f(i); print(x);
46
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Out of Thin Air
imagine this codeint x = 0, n = 0;
Thread 1 Thread 2 for (i=0; i<n; i++) x = 42; x += f(i); print(x);
can only print 42 in Java
47
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Out of Thin Air: Introduction of Writes
loop optimization in C/C++int x = 0, n = 0;
Thread 1 Thread 2 tmp = x; for (i=0; i<n; i++) x = 42; tmp += f(i);x = tmp; print(x);
48
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Out of Thin Air: Introduction of Writes
loop optimization in C/C++int x = 0, n = 0;
Thread 1 Thread 2 tmp = x; for (i=0; i<n; i++) x = 42; tmp += f(i);x = tmp; print(x);
can print 0 in C/C++
49
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Out of Thin Air
imagine this codeint x = 0, y = 0;
Thread 1 Thread 2 r1 = x; r2 = y;y = r1; x = r2;
50
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Out of Thin Air
imagine this codeint x = 0, y = 0;
Thread 1 Thread 2 r1 = x; r2 = y;y = r1; x = r2;
Expected result x == 0; y == 0;
Only possible result in Java
51
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Out of Thin Air: Optimization in C/C++
imagine this codeint x = 0, y = 0;
Thread 1 Thread 2 y = 42; r2 = y;r1 = x; x = r2;if (r1 != 42) y = r1;
Possible results in upcoming C++ MM x == 42; y == 42;
52
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Performance on Mult icore: Example
Single core application, 3 threads
All threads synchronize frequently on same lock
53
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Performance on Mult icore: Example
Single core application, 3 threads
All threads synchronize frequently on same lock
while (true) { synchronized (lock) { counter++; } doSomething(); }
54
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Single core application, 3 threads
Performance on Mult icore: Example
55
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Single core application, 3 threads
On a multicore
Performance on Mult icore: Example
56
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Performance on a Mult icore
Frequent synchronization can kill the performance
Typical non-RTOS will use heuristics to improve average performance
spin-lock for a short time
blocking for longer periods
57
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Performance on a Mult icore
These heuristics introduce priority-inversion and generally destroy predictability
A typical semaphore implementationdoes not take thread priority into account
does not limit worst-case-execution-time
58
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CPU Affinit ies
OSes provide APIs to lock threads certain CPUs
This limits the decision space of the RTOSa global scheduler for n CPUs would always run the n highest priority threads
with affinities, this may not be possible, e.g., if the two highest priority threads are locked to the same CPU
CPU affinities can introduce priority inversion!
So what are CPU affinities good for?
59
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CPU Affinit ies: No More interrupts
Locking interrupts to a dedicated CPUprotects all other CPUs from interrupts, and
invalidated cashes
WCETA is simplified considerably
60
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CPU Affinit ies: No More interrupts
Locking interrupts to a dedicated CPUprotects all other CPUs from interrupts, and
invalidated cashes
WCETA is simplified considerably
non-RT Task
Interrupt
CPU0
CPU1 A A A A A A A A A A A RT Task
61
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CPU Affinit ies: Separating Threads
Locking thread A to one CPU and threads B, C, ... to other CPUs may increase A's performance.
A will not be preempted by B, C, ..
A will not see its caches invalidated by B, C, ...
62
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CPU Affinit ies: Separating Threads
Locking thread A to one CPU and threads B, C, ... to other CPUs may increase A's performance.
A will not be preempted by B, C, ..
A will not see its caches invalidated by B, C, ...
non-RT Task
RT Task
CPU0
CPU1
CPU2
A AA A A A
B
B B
B
B
A A A A A
B
B
B
C C C
63
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CPU Affinit ies: Grouping Threads
Locking threads A and B to the same CPUwill increase shared memory communication between A and B
will avoid performance degradation on locking
will enable simple scheduling analysis (RMA)
64
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CPU Affinit ies: Grouping Threads
Locking threads A and B to the same CPUwill increase shared memory communication between A and B
will avoid performance degradation on locking
will enable simple schedule feasibility analysis (RMA)
RT Task ACPU0
CPU1
CPU2
BAA B A RT Task B
Other Task
C
C
DD D D
E E E E
E
E
EC C
GG
G
BA
65
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CPU Affinit ies: Diff icult Decisions
what if A and B both computation intensive and both access the same shared memory?
How can we use idle time?
...
66
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Multicore Scheduling for Realt ime
A pure priority based scheduler is not sufficient:
Imagine three tasks A, B, C on 2 CPUS
A
B
C
Release Deadline
67
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Multicore Scheduling for Realt ime
Priorities will cause deadline miss
A pri=10
B pri=10
CPU0
CPU1
C pri=9
Release Deadline
68
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Multicore Scheduling for Realt ime
Priorities will cause deadline miss
A pri=10
B pri=10
CPU0
CPU1
C pri=9Release Deadline
69
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Multicore Scheduling for Realt ime
CPU affinities do not help
A pri=10, {CPU0}
B pri=10, {CPU1}
CPU0
CPU1
C pri=9
Release Deadline
70
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Multicore Scheduling for Realt ime
Round robin could help
ACPU0
CPU1
C B
B A C B A C
A C B
Release Deadline
71
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
A C B
B A C B A C
A C B
Multicore Scheduling for Realt ime
Round robin could help
Release Deadline
CPU0
CPU1
72
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Performance on Multicore
Synchronization is expensive
Can synchronization be avoided?
Can lock free algorithms be used?Use compare and swap (CAS) instructions instead
73
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Lock Free Algorithms
Typical code sequence
do { x = counter; result = CAS(counter,x,x+1); }while (result != x);
74
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Compare-and-Swap Issues
Typical code sequence
do { x = counter; result = CAS(counter,x,x+1); }while (result != x);
What is the WCET? ∞?
75
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
On dual core:
# iterations
fre
qu
en
cy
Typical code sequence
do { x = counter; result = CAS(counter,x,x+1); }while (result != x);
What is the WCET? ∞?
Compare-and-Swap Issues
76
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Lock Free Library Code
Using libraries helpsAtomicInteger counter = new AtomicInteger();void increment(){ (void)counter.incrementAndGet();}
Code is easier and safer
Hand made lock free algorithms are not for normal application development
77
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Compare-and-Swap Solutions
One way state changes, without retry
Bounded number of retries
78
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
One Way State Changes Using CAS
Example codefor (i=0; i<N; i++) { new Thread() { public void run() { CAS(state,INIT,STARTING); [..] } }.start(); }
79
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Bounding Retries for CAS
introduce long enough code sections in between 2 compare-and-swap loops
then, if a retry is required, one other CPU was successful
after n-1 conflicts, one can be sure that all other CPUs are outside the CAS loop
80
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Bounding Retries for CAS: Example
AtomicInteger counter = new AtomicInteger(); static final int GRANULARITY = 64; [..] new Thread(){ int local_counter; public void incCounter() { local_counter++; if (local_counter >= GRANULARITY) { local_counter = 0; counter.addAndSet(GRANULARITY); } } }.start();
81
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
Measurements
alloc free sweep available Memory1E-1
1E+0
1E+1
1E+2
1E+3
1E+4
1E+5
1E+6
1E+7
1E+8
1E+9
1E+10
1E+11
123456789
Number of CAS tries is bounded:
On 8-CPU x86 system (Linux)
82
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists
List modifaction using CAS, single linked list
next
data
head next
data
next
data
A B C
83
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists
Add
next
data
head next
data
next
data
next
data
X
B CA B C
84
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists
Add
next
data
head next
data
next
data
next
data
X
B CA B C
85
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists
Add: CAS(head,A,X)
next
data
head next
data
next
data
next
data
X
B CA B C
86
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists
Add: CAS(head,A,X)
next
data
head next
data
next
data
next
data
A B C
X
87
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists
Add
next
data
head next
data
next
data
next
data
A B C
X
88
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists
Remove
head next
data
next
data
next
data
A B C
89
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists
Remove: CAS(head,A,B)
head next
data
next
data
next
data
A B C
90
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists
Remove: CAS(head,A,B)
head next
data
next
data
next
data
A B C
91
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists
Remove
head next
data
next
data
B C
92
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists, ABA Problem
Now, consider concurrent modifications:
next
data
head next
data
A B
next
data
C Thread 2Thread 1
93
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists, ABA Problem
next
data
head next
data
A B
next
data
C Thread 2Thread 1remove:CAS(head,A,B)
94
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists, ABA Problem
next
data
head next
data
A B
next
data
C Thread 2Thread 1remove:
CAS(head,A,B)
preempted
95
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists, ABA Problem
next
data
head next
data
A B
next
data
C Thread 2remove:CAS(head,A,B)
Thread 1remove:
CAS(head,A,B)
preempted
96
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists, ABA Problem
next
data
head next
data
A B
head next
data
B
next
data
C
next
data
C
Thread 2remove:CAS(head,A,B)
Thread 1remove:
CAS(head,A,B)
preempted
97
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists, ABA Problem
next
data
head next
data
A B
head next
data
B
next
data
C
next
data
C
Thread 2remove:CAS(head,A,B)
remove:CAS(head,B,C)
Thread 1remove:
CAS(head,A,B)
preempted
98
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists, ABA Problem
next
data
head next
data
A B
head next
data
B
next
data
C
next
data
C
head next
data
C
Thread 2remove:CAS(head,A,B)
remove:CAS(head,B,C)
Thread 1remove:
CAS(head,A,B)
preempted
99
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists, ABA Problem
next
data
head next
data
A B
head next
data
B
next
data
C
next
data
C
head next
data
C
Thread 2remove:CAS(head,A,B)
remove:CAS(head,B,C)
add A:CAS(head,C,A)
Thread 1remove:
CAS(head,A,B)
preempted
100
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists, ABA Problem
next
data
head next
data
A B
head next
data
B
next
data
C
next
data
C
head next
data
C
head next
data
C
next
data
A
Thread 2remove:CAS(head,A,B)
remove:CAS(head,A,B)
add A:CAS(head,C,A)
Thread 1remove:
CAS(head,A,B)
preempted
101
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists, ABA Problem
next
data
head next
data
A B
head next
data
B
next
data
C
next
data
C
head next
data
C
head next
data
C
next
data
A
head next
data
C
next
data
A
next
data
B
Thread 2remove:CAS(head,A,B)
remove:CAS(head,A,B)
add A:CAS(head,C,A)
Thread 1remove:
CAS(head,A,B)
preempted
102
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
CAS Lists, ABA Problem
next
data
head next
data
A B
head next
data
B
next
data
C
next
data
C
head next
data
C
head next
data
C
next
data
A
head next
data
C
next
data
A
next
data
B
Thread 2remove:CAS(head,A,B)
remove:CAS(head,A,B)
add A:CAS(head,C,A)
Thread 1remove:
CAS(head,A,B)
preempted
B was re-introduced!
103
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
ABA Problem: Solutions
Non solution: In C: free() after remove
Reason: newly allocated block may be sameSolution: In Java: only add new references
Reason: GC ensures old values no longer visibleSolution: Sync all threads before reuse
Example: Phase 1 moves elements from List1 to List2, Phase 2 moves elements back
Solution: Use 64-/128-bit CAS and mod. counter
104
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
ABA Solved via Modification Counter
next
data
head: 42 next
data
A B
next
data
C Thread 2Thread 1remove:
preempted
105
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
ABA Solved via Modification Counter
next
data
head: 42 next
data
A B
next
data
C Thread 2remove:CAS(head,A:42,
B:43)
Thread 1remove:
preempted
106
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
ABA Solved via Modification Counter
next
data
head: 42 next
data
A B
head: 43 next
data
B
next
data
C
next
data
C
Thread 2remove:CAS(head,A:42,
B:43)
Thread 1remove:
107
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
ABA Solved via Modification Counter
next
data
head: 42 next
data
A B
head: 43 next
data
B
next
data
C
next
data
C
head: 44 next
data
C
Thread 2remove:CAS(head,A:42,
B:43)
remove:CAS(head,B:43,
C:44)
Thread 1remove:
108
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
ABA Solved via Modification Counter
next
data
head: 42 next
data
A B
head: 43 next
data
B
next
data
C
next
data
C
head: 44 next
data
C
head: 45 next
data
C
next
data
A
Thread 2remove:CAS(head,A:42,
B:43)
remove:CAS(head,B:43,
C:44)
add A:CAS(head,C:44,
A:45)
Thread 1remove:
109
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
ABA Solved via Modification Counter
next
data
head: 42 next
data
A B
head: 43 next
data
B
next
data
C
next
data
C
head: 44 next
data
C
head: 45 next
data
C
next
data
A
Thread 2remove:CAS(head,A:42,
B:43)
remove:CAS(head,B:43,
C:44)
add A:CAS(head,C:44,
A:45)
Thread 1remove:
CAS(head,A:42,B:43)
retry!
110
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
ABA Solved via Modification Counter
next
data
head: 42 next
data
A B
head: 43 next
data
B
next
data
C
next
data
C
head: 44 next
data
C
head: 45 next
data
C
next
data
A
head: 46 next
data
C
Thread 2remove:CAS(head,A:42,
B:43)
remove:CAS(head,B:43,
C:44)
add A:CAS(head,C:44,
A:45)
Thread 1remove:
CAS(head,A:42,B:43)
retry!CAS(head,A:45,
C:46)
111
Multi-Core for Real-Time and Safety-Critical Software: Avoid the PitfallsDr. Fridtjof Siebert, CTO, aicas GmbHDr. James J. Hunt, CEO, aicas GmbH
ConclusionCode that runs well on single CPU may fail on multicore
Clear semantics of concurrent code is required for functional correctness
Cost of locking may be prohibitive
CPU affinities may help, but it is difficult to make the application more efficient
Lock free code is very hard to get right
A reliable memory model and good concurrent libraries are basis to avoid pitfalls.