A Mile-High View of Concurrent Algorithms Hagit Attiya Technion.
-
date post
19-Dec-2015 -
Category
Documents
-
view
218 -
download
2
Transcript of A Mile-High View of Concurrent Algorithms Hagit Attiya Technion.
A Mile-High View of Concurrent Algorithms
Hagit AttiyaTechnion
May 1, 2008 Concurrent algorithms @ COVA 2
A Simplistic View of Concurrent SystemsA collection of processes
Each a sequential thread of execution
Communicating through shared data structures
May 1, 2008 Concurrent algorithms @ COVA 3
Alternative Routes for Developing Concurrent AlgorithmsRefinement: Implementing high-level ADT from
lower-level ADTs– Safety conditions, liveness properties
– Hierarchical
Transactional: Support for running sequential applications concurrently– Safety conditions, liveness properties
May 1, 2008 Concurrent algorithms @ COVA 4
Abstract Data Types (ADT)
Cover most concurrent applications– At least encapsulate their data needs
– An object-oriented programming point of view Abstract representation of data
& set of methods (operations) for accessing it– Signature
– Specification
data
May 1, 2008 Concurrent algorithms @ COVA 5
data
data
Implementing High-Level ADT
May 1, 2008 Concurrent algorithms @ COVA 6
Implementing High-Level ADT
data
data
-------------------------------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------------------------------------------------------
Using lower-level ADTs
&procedures
May 1, 2008 Concurrent algorithms @ COVA 7
Lower-Level Operations
High-level operations translate into primitives on base objects– Obvious: read, write (restrictions?)– Common: compare&swap (CAS)– LL/SC, Double-CAS (2CAS,
DCAS), kCAS, …– Generic: read-modify-write (RMW), kRMW
Low-level operations are often implemented from more primitive operations– A hierarchy of implementations
May 1, 2008 Concurrent algorithms @ COVA 8
Executing Operations
P1
invocation response
P2
P3
May 1, 2008 Concurrent algorithms @ COVA 9
Interleaving Operations
Concurrent execution
May 1, 2008 Concurrent algorithms @ COVA 10
Interleaving Operations
)External (behavior
May 1, 2008 Concurrent algorithms @ COVA 11
Interleaving Operations, or Not
Sequential execution
May 1, 2008 Concurrent algorithms @ COVA 12
Interleaving Operations, or Not
Sequential behavior: invocations & response alternate and match (on process & object)
Sequential specification: All the legal sequential behaviors, satisfying the semantics of the ADT– E.g., for a (LIFO) stack: pop returns the last item pushed
May 1, 2008 Concurrent algorithms @ COVA 13
Correctness: Sequential consistency
[Lamport, 1979]
For every concurrent execution there is a sequential execution that– Contains the same operations
– Is legal (obeys the sequential specification)
– Preserves the order of operations by the same process
May 1, 2008 Concurrent algorithms @ COVA 14
Sequential Consistency: Examples
push(4)
pop():4push(7)
Concurrent (LIFO) stack
push(4)
pop():4push(7)
Last In First Out
May 1, 2008 Concurrent algorithms @ COVA 15
Sequential Consistency: Examples
push(4)
pop():7push(7)
Concurrent (LIFO) stack
Last In First Out
May 1, 2008 Concurrent algorithms @ COVA 16
Example 1: Multi-Writer Registers
Add logical time (Lamport timestamps) to values
Write(v)
read TS1,...,read TSn
TSi = max TSj +1
write v,TSi
Read only own value
Read()
read v,TSi
return v
Once in a while
read TS1,...,read TSn
write max TSj to TSi
~[Attiya, Welch TOCS 1994]
Using (multi-reader) single-writer registers
Need to ensure writes are eventually visible
May 1, 2008 Concurrent algorithms @ COVA 17
Multi-Writer Registers: Proof
Write(v,X)
read TS1,...,read TSn
TSi = max TSj +1
write v,TSi
Read(X)
read v,TSi
return v
Once in a while
read TS1,...,read TSn
write max TSj to TSi
Create sequential execution: – Place writes in timestamp order– Insert reads after the appropriate write
May 1, 2008 Concurrent algorithms @ COVA 18
Multi-Writer Registers: Proof
Create sequential execution: – Place writes in timestamp order– Insert reads after the appropriate write
Legality is immediate Per-process order is preserved since a read returns a
value (with timestamp) larger than the preceding write by the same process
May 1, 2008 Concurrent algorithms @ COVA 19
Sequential Consistency is not Composable
enq(Q1,X) enq(Q2,X) deq(Q1,Y)enq(Q2,Y) enq(Q1,Y) deq(Q2,X)
The execution is not sequentially consistent
May 1, 2008 Concurrent algorithms @ COVA 20
Sequential Consistency is not Composable
enq(Q1,X) deq(Q1,Y)enq(Q1,Y)enq(Q2,X)enq(Q2,Y) deq(Q2,X)
The execution projected on each object is sequentially consistent
Bad news forverification!
May 1, 2008 Concurrent algorithms @ COVA 21
Correctness: Linearizability
[Herlihy & Wing, 1990]
For every concurrent execution there is a sequential execution that– Contains the same operations
– Is legal (obeys the specification of the ADTs)
– Preserves the real-time order of non-overlapping operations
Each operation appears to takes effect instantaneously at some point between its invocation and its response (atomicity)
May 1, 2008 Concurrent algorithms @ COVA 22
Linearizable Multi-Writer Registers
Add logical time to values
Write(v,X)
read TS1,...,read TSn
TSi = max TSj +1
write v,TSi
Read(X)
read TS1,...,read TSn
return value with max TS
Using (multi-reader) single-writer registers[Vitanyi & Awerbuch, 1987]
May 1, 2008 Concurrent algorithms @ COVA 23
Multi-writer registers: Linearization orderCreate linearization:
– Place writes in timestamp order– Insert each read after the appropriate write
Write(v,X)
read TS1,...,read TSn
TSi = max TSj +1
write v,TSi
Read(X)
read TS1,...,read TSn
return value with max TS
May 1, 2008 Concurrent algorithms @ COVA 24
Multi-Writer Registers: Proof
Create linearization: – Place writes in timestamp order– Insert each read after the appropriate write
Legality is immediate Real-time order is preserved since a read returns a value
(with timestamp) larger than all preceding operations
May 1, 2008 Concurrent algorithms @ COVA 25
Linearizability is Composable
The whole system is linearizable each object is linearizable
Allows to implement and verify each object separately
Good news forverification!
May 1, 2008 Concurrent algorithms @ COVA 26
Example 3: Atomic Snapshot
m components Update a single component Scan all the components
“at once” (atomically)
Provides an instantaneous view of the whole memory
update
ok
scan
v1,…,vm
May 1, 2008 Concurrent algorithms @ COVA 27
Atomic Snapshots: Algorithm
Update(v,k)
A[k] = v,seqi,iScan()
repeat
read A[1],…,A[m]
read A[1],…,A[m]
if equal
return A[1,…,m]
Linearize:• Updates with their writes• Scans inside the double collects
double collect
[Afek, Attiya, Dolev, Gafni, Merritt, Shavit, JACM 1993]
May 1, 2008 Concurrent algorithms @ COVA 28
Atomic Snapshot: LinearizabilityDouble collect (read a set of values twice)
If equal, there is no write between the collects– Assuming each write has a new value (seq#)
Creates a “safe zone”, where the scan can be linearized
read A[1],…,A[m] read A[1],…,A[m]
write A[j]
May 1, 2008 Concurrent algorithms @ COVA 29
Liveness Conditions (Eventual)
Wait-free: every operation completes within a finite number of (its own) steps no starvation for mutex
Nonblocking: some operation completes within a finite number of (some other process) steps deadlock-freedom for mutex
Obstruction-free: an operation (eventually) running solo completes within a finite number of (its own) steps– Also called solo termination
wait-free nonblocking obstruction-free
May 1, 2008 Concurrent algorithms @ COVA 30
Liveness Conditions (Bounded)
Wait-free: every operation completes within a bounded number of (its own) steps no starvation for mutex
Nonblocking: some operation completes within a bounded number of (some other process) steps deadlock-freedom for mutex
Obstruction-free: an operation (eventually) running solo completes within a bounded number of (its own) steps– Also called solo termination
Bounded wait-free bounded nonblocking bounded obstruction-free
May 1, 2008 Concurrent algorithms @ COVA 31
Wait-free Atomic Snapshot[Afek, Attiya, Dolev, Gafni, Merritt, Shavit, JACM 1993]
Embed a scan within the Update.
Update(v,k)
V = scan
A[k] = v,seqi,i,V
Scan()
repeat
read A[1],…,A[m]
read A[1],…,A[m]
if equal
return A[1,…,m]
else record diff
if twice pj
return Vj
Linearize:• Updates with their writes• Direct scans as before• Borrowed scans with source
direct scan
borrowedscan
May 1, 2008 Concurrent algorithms @ COVA 32
Atomic Snapshot: Borrowed ScansInterference by process pj
And another one… pj does a scan inbeteween
Linearizing with the borrowed scan is OK.
write A[j]
read A[j]… …
read A[j]… …
embedded scan write A[j]
read A[j]… …
read A[j]… …
May 1, 2008 Concurrent algorithms @ COVA 33
Alternative Routes for Developing Concurrent AlgorithmsRefinement: Implementing high-level ADT from
lower-level ADTs– Safety conditions, liveness properties
– Hierarchical
Transactional: Support for running sequential applications concurrently– Safety conditions, liveness properties
May 1, 2008 Concurrent algorithms @ COVA 34
The Transactional Approach
[Herlihy, Moss, ISCA 1993]
Systematic approach for implementing concurrent data structures
Program sequentially But run concurrently Should appear to execute sequentially No high-level signature / semantics
– The sequential program is the specification
May 1, 2008 Concurrent algorithms @ COVA 35
Transactional Synchronization
A transaction aggregates a sequence of resource accesses to be executed atomically– A sequence of atomic actions
– Like in database systems A transaction ends either
by committing– all its updates take effect
or by aborting– no update is effective
Read XWrite XRead ZRead Y
Read XWrite XRead ZRead Y
May 1, 2008 Concurrent algorithms @ COVA 36
Safety: Serializability
[Papadimitriou 79][Weikum, Vossen, 2002, Chapter 3]
An analogue of sequential consistency Any interleaving of the transactions yields a
result that could be achieved in a sequential execution of the same set of transactions (a serialization)– Just the committed transactions?
(Aborted transactions have no effect.) Final state serializability
May 1, 2008 Concurrent algorithms @ COVA 37
Safety: View Serializability
[Yannakakis 1984]
What about intermediate values read?– Could be “corrected” later– But still cause harm, e.g., division by 0
Any interleaving of the transactions has an equivalent sequential execution of the same transactions– Where all reads return the same value
Makes no sense in the context of implementing a high-level ADT– Where the internals are not exposed
May 1, 2008 Concurrent algorithms @ COVA 38
Safety: Strict Serializability
[Papadimitriou 79][Bernstein, Shipman & Wong, 1979]
The serialization must preserve the real-time order of (non-overlapping) transactions An analogue of linearizability Called ordered serializabililty in W&V
Strict view serializability
May 1, 2008 Concurrent algorithms @ COVA 39
Opacity
[Guerraoui & Kapalka, PPoPP 08]
Essentially, strict view serializability – Applied to all transactions
(not only committed or completed ones)
– Generalized to arbitrary object types (not just reads and writes)
May 1, 2008 Concurrent algorithms @ COVA 40
Liveness Properties
As in high-level implementations But restricted to successful completions
(commit)
E.g., wait-freedom every transaction eventually commits,obsruction-freedom every transaction (eventually) running solo terminates
May 1, 2008 Concurrent algorithms @ COVA 41
NonblockingObstruction-free
Obstruction-free
A Shift in Terminology
< 2003 ≥ 2003
Lock-free
Nonblocking
Wait-free
Lock-free
Wait-free
May 1, 2008 Concurrent algorithms @ COVA 42
Where’s the Confusion?
[Herlihy & Wing, TOPLAS 1990]
[Herlihy, TOPLAS 1991]
[Herlihy, Luchangco, Moir, ICDCS 2003]
May 1, 2008 Concurrent algorithms @ COVA 43
The Road Ahead
Concurrent algorithms pose a great challenge
It is “easy” to write – Correct algorithms and let
efficiency take care of itself
– Efficient algorithms and let correctness take care of itself
But very hard to write correct & efficient algorithms
Systematically…