Post on 08-Jul-2018
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 1/55
1
Synchronization & clocks
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 2/55
2
Synchronization
Synchronization: coordination of actionsbetween processes.
Processes are usually asynchronous,
(operate without regard to events in otherprocesses)
Sometimes need to cooperatesynchronize
!or mutual e"clusion !or event ordering (was message " from
process P sent before or after message y fromprocess #$)
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 3/55
3
Synchronization in centralized systems isprimarily accomplished through sharedmemory
%vent ordering is clear because all events aretimed by the same cloc&
Synchronization in distributed systems isharder
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 4/55
4
Fundamental Limitations of aDistributed System
'istributed systems are assumed tohave no global cloc&
'istributed systems have no sharedmemory
hese limitations force us to be creativein solving the following fundamental
problems rdering events
*apturing a global state
%nsuring mutual e"clusion
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 5/55
5
Model
Distributed system model: n processes connected by a communications
networ&
imperfectly synchronized cloc&s, thus noe"act notion of +what time it is at otherprocesses
reliable message delivery
may or may not bound message deliverytime
no shared memory
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 6/55
6
Time in Distributed Systems
ime is an important and interesting issue indistributes systems. %"ample: e-commerce transaction timestamp for
auditing and accountability purposes. eed time for maintaining consistency of databases.
hree notions of time: ime seen by e"ternal observer: / global cloc& of
perfect accuracy ime seen on cloc&s of individual hosts: %ach has its
own cloc&, and cloc&s may drift out of sync 0ogical time: event a occurs before event b and this is
detectable because information about a may havereached b b is causally dependent on a
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 7/55 7
Logical Time
/bstraction 'oesnt provide a cloc& in the sense of wall
cloc& time
!ocus is on definition of the +happens beforerelationship between events +*ould whats happening in this event have
been influenced by what happened in that event$
%vents are: message sends message receptions local events
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 8/55
8
Logical Clocks
bservation: if two processes (running onseparate processors) do not interact, it doesntmatter if their cloc&s are not synchronized.
bservation: 2hen processes do interact, theyare usually interested in event order , instead ofe"act event time.
3ence 0ogical cloc&s are sufficient for manyapplications
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 9/55
9
Logical Time: The Picture
a
c
e
d
p0
p1
p2
p3
a,b and a,c are concurrent
c happens after b
e happens after a, b, c, d
b
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 10/55
10
a!!ened"#efore $elation
he happened before relation ( ) captures thecausal relationships between events: a b if a and b are events in the same process
and a occurred before b
a b if a is the send event of a message m and b is the corresponding receive event
is a transitive relation %vent a potentially causally affects event b if a b %vent a is concurrent with event b if neither
a b nor b a
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 11/55
11
Three %m!ortant Schemes
0amports logical cloc&s cloc& is a single integer
4ector cloc&s cloc& is a vector of integers with length n n 5 number of processes
6atri" cloc&s cloc& is an n " n matri" of integers n 5 number of processes
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 12/55
12
Lam!orts Clocks
0amports cloc&s captures partialordering defined by the relation
0amports logical cloc&s: %ach process i maintains a counter Ci whose
current value is attached to each message sent
7nternal events in a process cause the counterCi to be incremented
2hen a message is received by a process i , Ci isset to ma"(m.Cj, Ci) + 1
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 13/55
13
3appened before relation: a -8 b : %vent a occurred before event b. %vents in
the same process.
a -8 b : 7f a is the event of sending a message min a process and b is the event of receipt of thesame message m by another process.
a -8 b, b -8 c, then a -8 c. +-8 is transitive.
*ausally rdered %vents a -8 b : %vent a +causally affects event b
*oncurrent %vents nly one is true: a -8 b (or) b -8 a (or) a 99 b.
Lam!orts Clock
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 14/55
14
S!ace"time Diagram
Time
Space P1
P2
e11 e12 e13 e14
e21 e22 e23 e24
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 15/55
15
Logical Clocks *onditions satisfied:
*i is cloc& in Process Pi.
7f a -8 b in process Pi, *i(a) *i(b)
a: sending message m in Pi; b : receiving message m in P<;then, *i(a) *<(b).
7mplementation =ules: R1: Ci = Ci + d (d > 0); cloc& is updated
between two successive events at a process.
R2: Cj = max(Cj, tm + d); (d > 0); 2hen P<receives a message m with a time stamp tm
(tm assigned by Pi, the sender; tm 5 *i(a), abeing the event of sending message m).
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 16/55
16
S!ace"time Diagram
Time
Space P1
P2
e11 e12 e13 e14
e21 e22 e23 e24
e15 e16 e17
e25
(1) (2) (3) (4) (5) (6) (7)
(1) (2) (3) (4) (7)
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 17/55
17
Limitation of Lam!ortsClock
Time
Space P1
P2
e11 e12 e13 e14
e21 e22 e23 e24
P3 e31 e32 e33
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 18/55
18
Limitation of Lam!orts Clock
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 19/55
19
Lam!orts Clocks: Total 'rder
defines a partial order rather than a totalorder
3ow to impose a total order$
>se process 7's to brea& ties 7mportant: otal order is +artificial--it has
nothing to do with the cloc& on the wall?
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 20/55
20
(ector Time
4ector time correctly captures the causalityrelationship between distributed events (capturesthe transitive closure of the relation)
%ach process i maintains a vector of state numbers
Si to &eep trac& of transitive dependencies amongprocesses Si@1..nA: is a +vector cloc& at process Pi whose
entries are the +assumedbest guess cloc& valuesof different processes.
Si@<A (< 8i) is the best guess of Pi for P<s cloc&. Si@iA contains the current state number for process
i; Si@nA, n ≠ i, contains the most recent statenumber of process n upon which i currently depends
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 21/55
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 22/55
22
(ector Time: The Picture
p0
p1
p2
p3
1 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
1 0 0 0
1 2 0 0 1 3 2 0 1 4 2 0
1 4 2 00 1 0 0
0 1 1 0
0 1 0 0
0 1 2 0
0 1 2 0
1 4 2 1
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 23/55
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 24/55
24
#irman"Schi!er"Ste!hensonProtocol
(i) o broadcast m from process i, increment *i(i),and timestamp m with 4m 5 *i@iA
(ii) 2hen < G i receives m, < delays delivery of muntil
1. * <@iA 5 4m@iA E1 andC. * <@&A H 4m@&A for all & G i
1. says m is the ne"t msg P < was e"pecting from Pi
C. says P < has seen all msgs seen by Pi when Pi sentm.'elayed messaged are Iueued in < sorted byvector time. *oncurrent messages are sorted byreceive time. 2hen m is delivered at <, * < isupdated according to vector cloc& rule
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 25/55
25
Causal Messageordering-contd.
" ot#er $ords, i% a messa&e m is re'eived%rom i, you s#ould also #ave re'eived every
messa&e t#at P i re'eived be%ore it set m;
e&,
if m is sent by P 1 and ts(m) is (J, K, L) and
you are P 3, you should already have received
e"actly C messages from P1 and at least K
from PC
if m is sent by PC and ts(m) is (K, M, 1, J)
and if you are PJ and 4*J is (J, J, K, J) then
you need to wait for a fourth message from
PC and at least one more message from P1.
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 26/55
26
P0
P1
P2
(1, 0, 0)
P1 received message m from P0 before sending
messagem*
to P2; P2 must wait for delivery ofm
before receiving m*
(ncrement own cloc! only on message send)
"efore sending or receiving any messages, one#s
own cloc! is (0, 0, $0)
%&2
(1, 0, 0) (1, 1, 0)
(1, 1, 0)%&1
m
m'
%&0
%&2
nforcing &ausal &ommunication
%&0
(1, 1, 0)
(0, 0, 0)%&2
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 27/55
27
(ector Time: Com!arison
4ector timestamps are compared in thefollowing manner:– TSi ! TS" i#
∀ k, TSi[k] ≤ TS"[k] and k, TSi[k] ! TS"[k].
eg @ (J,J,M) (J,K,M) A
– $n event e causa%%& de'ends on an evente) i#
e).TS ! e.TS
– Two events e and e) are concurrent
i* neither e).TS ! e.TS nor e.TS ! e).TS
∃
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 28/55
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 29/55
29
Matri/ Clocks
6otivation 6y vector cloc& describes what 7 +see
7n some applications, 7 also want to &now what other people see
6atri" cloc&: %ach event has n vector cloc&s, one for each process
he i th vector on process i is called process i s principle vector i.e. its own vector cloc& as before
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 30/55
30
Matri/ Time
7n a system of matri" cloc&s, the time is representedby a set of n N n matrices of non-negative integers.
/ process pi maintains a matri" mti @1..n, 1..nA where, mti*i , i denotes the local logical cloc& of pi and
trac&s the progress of the computation at processpi . mti*i , j denotes the latest &nowledge that process
pi has about the local logical cloc&,mt <@< , < A, of process p< .
mti*j , represents the &nowledge that process pihas about the latest &nowledge that p< has about thelocal logical cloc&, mt& @&, &A, of p&
he entire matri" mti denotes pi s local view of theglobal logical time.
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 31/55
31
Process pi uses the following rules =1 and =C to update itscloc&:
=1 : Oefore e"ecuting an event, process pi updates its locallogical time as follows:
mti*i , i := mti*i , i + d (d > 0)
=C: %ach message m is piggybac&ed with matri" time mt.2hen pi receives such a message (m,mt) from a process p<, pi e"ecutes the following seIuence of actions:
>pdate its global logical time as follows:
(a) 1 -= -= : mti *i , := max(mti*i , , mt*j , )
(hat is, update its row mti @i , A with the p< s row in thereceived timestamp,mt.)
(b) 1 -=,l -= : mti*, l:= max(mti*, l , mt*, l )
%"ecute =1. 'eliver message m.
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 32/55
32
.atri/ &loc! /amle
user1 (rocess1)
user2 (rocess2)
user3 (rocess3)
(1,0,0)
(0,0,0)
(0,0,0)
(0,0,0)
(0,1,0)
(0,0,0)
(0,0,0)
(0,0,0)
(0,0,1)
(2,0,0)
(0,0,0)
(0,0,0)
(3,0,0)
(0,0,0)
(0,0,0)
(0,0,0)
(0,0,0)
(0,0,2)
(0,0,0)
(0,2,2)
(0,0,2)
(2,0,0)
(2,3,2)
(0,0,2)
(2,0,0)
(2,3,2)
(2,3,3)
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 33/55
33
Pro!erty of matri/ clocks:
7f min& (mti@&, l A) H t
hen process pi &nows that every other process p&
&nows that pl s local time has progressed till t.
7f this is true, it is clear that process p i &nowsthat all other processes &now that pl will never
send information with a local time t.
7n many applications, this implies that processes
will no longer reIuire from pl certain informationand can use this fact to discard obsoleteinformation.
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 34/55
34
0hy the need for global staterecording,
2hen it is &nown that local computations havestopped and that there are no more messages intransit, the system has obviously entered a statein which no more progress will be made.
deadloc&ed$
correctly terminated$
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 35/55
35
1lobal state of a distributedsystem
/bsence of global cloc& leads to a problemin global state recording
2hat is global state$ 0ocal state of each process and
he state of communication channels i.e.
messages that are currently in transit(sent but not received)
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 36/55
36
1lobal State
$500 $200
AB
C1: Empty
C2: Empty
G!"a State 1
$450 $200
A B
C1: T# $50
C2: Empty
G!"a State 2
$450 $250
A B
C1: Empty
C2: Empty
G!"a State 3
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 37/55
37
o* to record the global state
'istributed snapshot reflects a state in which the distributed systemmight have been
reflects a consistent global state 7f we have recorded that process P has
received a msg from another process #, thenwe should also have recorded that process #had actually sent the msg
he reverse condition (# has sent a msg that Phas not yet received) is allowed.
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 38/55
38
Cut2
/ 'ut represents the last event that hasbeen recorded for each of severalprocesses.
/ll recorded msg receipts have a correspondingrecorded send event
/n inconsistent cut would have a receipt ofa msg but no corresponding send event
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 39/55
39
3/am!leProducer Consumer !roblem
p records its state
m
p
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 40/55
40
3/am!le
p
m
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 41/55
41
3/am!le
I records its state
p
m
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 42/55
42
3/am!leThe recorded state
m
p
m
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 43/55
43
0here did *e err,
2hat did we do$
p
m
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 44/55
44
3rror22
he sender has no record of the sending
he receiver has the record of the receipt
=esult Qlobal state has record of the receive event
but no send event violating the happenedbefore concept??
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 45/55
45
$ecording 1lobal State
Send(6i<): message 6 sent from Si to S<
rec(6i<): message 6 received by S<, from Si
time("): ime of event "
0Si: local state at Si send(6i<) is in 0Si iff (if and only if)
time(send(6i<)) time(0Si)
rec(6i<) is in 0S< iff time(rec(6i<)) time(0S<)
transit(0Si, 0S<) : set of messagessentrecorded at 0Si and receivedrecorded at 0S<
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 46/55
46
$ecording 1lobal State 4
inconsistent(0Si,0S<): set of messages sentrecorded at 0Si and receivedrecorded at0S<
Qlobal State, QS: D0S1, 0SC,R., 0SnF
*onsistent Qlobal State, QS 5 D0S1, ..0SnF/' for all i in n, inconsistent(0Si,0S<) is null.
ransitless global state, QS 5 D0S1,R,0SnF/' for all 7 in n, transit(0Si,0S<) is null.
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 47/55
47
$ecording 1lobal State 55
S1
S2
%1 %2&S1
&S2
%1: t'ait
%2: ic!itet
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 48/55
48
$ecording 1lobal State555
Strongly consistent global state: consistentand transitless, i.e., all send and thecorresponding receive events are recorded inall 0Si.
&S11
&S21
&S31
&S12
&S22 &S23
&S32 &S33
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 49/55
49
Chandy"Lam!ort 6lgorithm
'istributed algorithm to capture a consistentglobal state. *ommunication channels assumed tobe !7!.
>ses a marer to initiate the algorithm. Sending 6ar&er by P:
P records its state. !or each outgoing channel *, Psends a mar&er on * with the state info.
=eceiving 6ar&er by #: 7f # has recorded its state: a. =ecord the state as
an empty seIuence, S%' mar&er (use above rule).
%lse (# has recorded state before): =ecord the state of* as seIuence of messages received along *, after #sstate was recorded and before # received the mar&er.
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 50/55
50
Chandy"Lam!ort 6lgorithm
7nitiation of mar&er can be done by anyprocess, with its own uniIue mar&er:process id, seIuence number8.
Several processes can initiate state recording
by sending mar&ers. *oncurrent sending ofmar&ers allowed.
ne possible way to collect global state: allprocesses send the recorded state information
to the initiator of mar&er. 7nitiator process cansum up the global state.
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 51/55
51
6lgorithm in 6ction
p
S
0 S1 S
2 S3
S p0 S p
1 S p2 S p
3
m1 m2 m3
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 52/55
52
6lgorithm in 6ction
p
S
0 S1 S
2 S3
S p0 S p
1 S p2 S p
3
m1 m2 m3
'ec!'* tate a S1 + e* ma',e' t! p
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 53/55
53
6lgorithm in 6ction
p
S
0 S1 S
2 S3
S p0 S p
1 S p2 S p
3
m1 m2 m3
p 'ec!'* tate a S p2+ c-ae tate a empty
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 54/55
54
6lgorithm in 6ction
p
S
0 S1 S
2 S3
S p0 S p
1 S p2 S p
3
m1 m2 m3
'ec!'* c-ae tate a m3
8/19/2019 Clocks and Global State.ppt
http://slidepdf.com/reader/full/clocks-and-global-stateppt 55/55
6lgorithm in 6ction
p
S
0 S1 S
2 S3
S p0 S p
1 S p2 S p
3
m1 m2 m3
.ec!'*e* G!"a State / ((S p2+ S
1)+ (0+m3) )