CSS434 Time & Global States1 CSS434 Time and Global States Textbook Ch11 Professor: Munehiro Fukuda.

CSS434 Time & Global States 1

CSS434 Time and Global States CSS434 Time and Global States Textbook Ch11Textbook Ch11

Professor: Munehiro Fukuda


Outline Physical clock synchronization

Applications: make Logical clock synchronization

Parallel and distributed simulation Global states and consistent cuts

Distributed garbage collection, deadlock detection, distributed termination detection, and discrete-even simulation

Distributed debugging Checking if a captured snapshot is one of

transitory states we have considered.


Why Clock Synchronization

Computer clock: a counter decremented by a crystal oscillation. Single computers: all processes use the same clock. – No

problem Multiple computers: impossible to guarantee that the

crystals in different computers all run at exactly the same frequency.

Synchronization: Absolute (with real time)

Necessary for real-time applications such as on-line reservation systems

Relative (with each other) Required for those applications that need a consistent view

of time across all nodes.

Network


Clock SynchronizationPassive Centralized Algorithms – Christian’s

Algorithm

Assumption: processing time has been measured or estimated

Message_delay = (T1 – T0 – processing)/2 New client time = T + message_delay Improvements:

Average multiple measurements Discard outlying measurements

Client

Time serverTime

T0 T1

Time? Time=T

Processing


Clock SynchronizationActive Centralized Algorithm – Berkeley

Algorithm

Assumption: processing time has been measured or estimated Server: diff(i) = server_time – (ci_time + message_deley) Client: ci_time = ci_time + diff(i)

Time server

Client 1

Client 2

C1_time

C2_time

Time?

Time?

Diff(1)

Diff(2)


Clock SynchronizationDistributed Algorithm – Averaging Algorithm

Assumption: R is large enough to wait for all broadcast messages All nodes broadcast their time periodically Each node computes average. Improvement:

Discard outlying time messages. Exchange their time with their local neighbors.

Node 1

Node 2

Node 3

T0 T0 +R T0 +2R T0 +3R

N3_time=30

N3_time=30N2_time=32

N2_time=32

N1_time=31

N1_time=31


Clock SynchronizationNetwork Time Protocol

1

2

3

2

3 3

Note: Arrows denote synchronization control, numbers denote strata.

UDP message

UTC (Coordinated Universal Time)


Clock SynchronizationNetwork Time Protocol

Ti

Ti-1Ti -2

Ti- 3

Server B

Server A

Time

m m'

Time

tt’

di = t + t’ = Ti-2 – Ti-3 + Ti – Ti-1

2o = Ti-2 – Ti-3 – t + Ti-1 – Ti + t’o = (Ti-2 – Ti-3 + Ti-1 – Ti)/2 + (t’ – t)/2

Ti-2 = Ti-3 + t + o

o

Ti = Ti-1 + t’ - o


Event OrderingHappened-Before Relation

1. Event 　 eki: The kth event of process i

2. Sequence 　 hki ： The history of process I through

to the event eki

3. Cause-and-effect 　 e→e’: e proceeds e’.4. Parallel events 　 e e’: e and e’ happen in parallel∥5. Happens-Before Relation:

1. If eki, el

i h∈ i and k < l, then eki → el

i,2. If ei = send(m) and ej = receive(m), then ei → ej,3. If e → e’ and e’ → e”, then e → e”

Most applications need not maintain the real-timesynchronized clock.


Event OrderingLogical Clock

1. ee’ LC(e) < LC(e’) for all events2. However, we cannot inferLC(e) < LC(e’) ee’

Example: LC(e21) > LC(e1

3) but e21 || e1

3

LC(ei) := (ei != receive(m)) ? LC + 1 : max(LC, TS(m)) + 1where TS(m) is the timestamp of message m:

P1

P2

P3

e11 e2

1

e12 e2

2

e13 e2

3

LC=1 LC=2

LC=3 LC=4

LC=1 LC=5

m1

m1


Event OrderingVector Clock

P1

P2

P3

e11 e2

1

e12 e2

2

e13 e2

3

m1

m1

(1,0,0) (2,0,0)(2,1,0) (2,2,0)

(0,0,1) (2,2,2)

Vi[I] = vi[i] + 1;Pi includes the value t = Vi in every message it sendsVi[j] = max(vi[j], t[j]) for j = 1,2,…,N

1. ee’ V(e) < V(e’)2. V(e) < V(e’) ee’

Example:neither V(e21) V(e1

3) nor V(e21) V(e1

3), and thus e21 || e1

3


p2p1

message

garbage object

objectreference

a. Garbage collection

p2p1 wait-for

wait-forb. Deadlock

p2p1

activatepassive passivec. Termination

Global StateApplications necessary to detect a correct global

state

This is not a garbage, becausethe in-transit message points to it

Both p1 and p2 are passive and thus seems ready to finish, but an in-transit message makes p1 active again.


Global StateConsistent Cut

Finding C such that (e C) (e’ → e) e’ C∈ ∧ ⇒ ∈

p1p1

p2p2

p3p3

p4p4

ee1111

ee1133

ee1122

ee2222 ee33

22

ee2211 ee22

11

ee2233

ee1144 ee22

44ee33

44

CC C’C’

(send)(send)

(receive)(receive)


Global StateDistributed Snapshot – Chandy/Lamport [1985]

A process that wants to take a snapshot sends a snapshot request to the others. Each process records its state upon receiving the first snapshot request. Each process keep recording the messages until receiving a snapshot request

from each of the other process except the one that has originally initiated a snapshot.

P0

P1 P2

P0

P1

P2

Snapshot request

Ordinary message

Message recording

s

mm m s

ss

s


Marker (Snapshot request) receiving rule for process pi

On pi’s receipt of a marker (snapshot request) message over channel c:if (pi has not yet recorded its state) it

records its process state now;records the state of c as the empty set;turns on recording of messages arriving over other incoming channels;

else pi records the state of c as the set of messages it has received over c since it saved its state.

end ifMarker (Snapshot request) sending rule for process pi

After pi has recorded its state, for each outgoing channel c: pi sends one marker message over c (before it sends any other message over c).

Global StateDistributed Snapshot – Chandy/Lamport [1985]


A Distributed Snapshot Example

p1 p2(empty)<$1000, 0> <$50, 2000>

(empty)

c2

c1

1. Global state S0

2. Global state S1

3. Global state S2

4. Global state S3

p1 p2(Order 10, $100), M<$900, 0> <$50, 2000>

(empty)

c2

c1

p1 p2(Order 10, $100), M<$900, 0> <$50, 1995>

(five widgets)

c2

c1

p1 p2(Order 10, $100)<$900, 5> <$50, 1995>

(empty)

c2

c1

(M = marker message)

Time line

Consistent Cut

Start recording its state

Record its stateMStop recording


Samadi’s Algorithm [1985]

p2p2

p1p1

p3p3

p0p0Take snapshotTake snapshot

16161212

1515

tagtag

Report 15Report 15Report 12Report 12

2020

ackack

Report 20Report 20

1. Each process returns an ack whenever receiving a message.2. Once receiving a snapshot message, each process returns a tag instead of an ack until a new

GVT is compute.3. When receiving a snapshot message, each process returns to P0 the minimum time among:

- the minimum timestamp among events that have not yet been processes. - the minimum timestamp among messages that have not yet been acknowledged.- the minimum timestamp among tags it has received.

2020

Done


Mattern’s Algorithm [1993]

p2p2

p1p1

p3p3

p4p4(0,0,0,0)(0,0,0,0) (0,1,0,0)(0,1,0,0)

(0,2,-1,0)(0,2,-1,0)

(0,0,0,1)(0,0,0,1)

(0,0,1,1)(0,0,1,1)

(0,0,1,0)(0,0,1,0)

(0,0,0,0)(0,0,0,0)

1st snapshot1st snapshot

2nd snapshot2nd snapshot

1. Process Pi maintains a vector counter: Vi[1..n].2. Pi writes in Vi[j] the number of messages sent to Pj.3. Pi subtract one from Vi[j] when receiving a message from Pj4. During the 1st circulation of a ‘take snapshot’ message, Pi performs:

C[1..n]+=Vi[1..n]; Vi[1..n] = 0Upon completing the 1st circulation, c[I] presents the number of messages in transit to Pi.

5. During the 2nd circulation, Pi wait for performs:C[i] = 0

-1 +1 -1

+1

+1

+1

-1

-1+1 -1


An Example: Parallel and Distributed Simulation

1cell/time unit1cell/5 time units

1cell/20 time units

Process 0

Process 3 Process 2

Process 1


An Example: Parallel and Distributed Simulation (Cont’d)

Process 0

Process 1

Process 2

Process 3

e2

e1e5 e6

e4e3

e7

e11e10e9e8 e12

e1

e1

attack

attack

Barrier per every simulation cycle

What drawbacks does this method have?


An Example: Parallel and Distributed Simulation (Cont’d)

Process 0

Process 1

Process 2

Process 3

e2

e1e5 e6

e4e3

e7

e11e10e9e8 e12

e1

e1

attack

attack

Discrete event simulation with optimistic synchronization

e2 e3

e13 e14

Rollback

Rollback

Old event history kept so as to rollback computation.

When can we garbage collect such history?


Time Warp[Jefferson 1985]Optimistic Distributed Simulation• Each process has an input message, an output message, and an event history queue.• When a process receives a message whose timestamp is older than its local time:

　１． Roll back its local event execution to that old timestamp.　２． Roll back its receipt of input messages whose timestamp is newer than that old

timestamp.３． Send anti-messages to cancel all emanated messages whose timestamp is newer than that old timestamp.

• GVT (Global Virtual Time): is periodically computed to garbage-collect all the executed events whose timestamp is older than GVT.

152152p2p2

p1p1

p3p3 163163

142142

143143

162162142142141141

141141120120

120120

122122

121121

Arrived lateArrived late

135135

RollbackRollback

Anti-messageAnti-message

LVTLVT

LVTLVT

LVTLVT


SPEEDS[Steinman 1992]Breathing Time Buckets

p2p2

p1p1

P1’s LEHP1’s LEH

P2’s LEHP2’s LEHNext GEHNext GEH

(GVT)(GVT)

This is an optimistic distributed simulator, but so aggressive as Time Warp.• Each process broadcasts the oldest local even among those it will execute. This is called a

Local Event Horizon (LEH).• A process must suspend its even processing if it has received an older LEH than the one it

is currently processing.• The oldest LEH among all processes become the next Global Event Horizon (GEH).• Each process may send out all messages and process all events before this new GEH.• Processes which have already processed beyond GEH must roll back their computation to

GEH. No anti-messages are sent out.


Vector timestamps and variable values for the execution

m1 m2

p1

p2Physical

time

Cut C1

(1,0) (2,0) (4,3)

(2,1) (2,2) (2,3)

(3,0)x1= 1 x1= 100 x1= 105

x2= 100 x2= 95 x2= 90

x1= 90

Cut C2

Constraints |x1 – x2| <= 50. Before this is violated, a process must send its value toits partner.


The lattice of global states for the execution

Sij= global state after i events at process 1 and j events at process 2

S00

S10

S20

S21S30

S31

S32

S22

S23

S33

S43

Level 0

1

2

3

4

5

6

7


Evaluating possibly and definitely

F = (φ(S) = );False = (T φ(S) = ) True

?

–

0Level

1

2

3

4

5

F

F

F

F T

F

m1 m2

p1

p2Physical

time

Cut C1

(1,0) (2,0) (4,3)

(2,1) (2,2) (2,3)

(3,0)x1= 1 x1= 100 x1= 105

x2= 100 x2= 95 x2= 90

x1= 90

Cut C2

All included: (2,2) = True

Possibility

Must be True, and thud we don’t care

F

Definitely = T for all linialization(4,3) = T and this is definitely


Paper Review by Students Distributed Snapshot

Samadi’s Algorithm Mattern’s Algorithm Discussions: What are pros & cons of these

algorithms? Optimistic Synchronization

SPEEDS Time Warp Discussions: What are pros & cons of these

algorithms in terms of performance, process creation/termination, dynamic memory allocation, and I/O handling?


Exercises (No turn-in)1. Textbook p627, Q14.7: An NTP server B receives server A’s message at

16:34:23.480 bearing a timestamp 16:34:13.430 and replies to it. A receives the message at 16:34:15.725, bearing B’s timestamp 16:34:25.7. Estimate the offset between B and A and the accuracy of the estimate.

2. Textbook p628, Q14.14: Two processes P and Q are connected in a ring using two channels, and they constantly rotate a message m. At any one time, there is only one copy of m in the system. Each process’s state consists of the number of times its has received m, and P sends m first. At a certain point, P has the message and its state is 101. Immediately after sending m, P initiates the snapshot algorithm. Explain the operation of the algorithm in this case, given the possible global state(s) reported by it.

3. Textbook p429, Q14.15: The figure below shows events occurring for each of two processes, p1 and p2. Arrows between processes denote message transmission. Draw and label the lattice of consistent states (p1 state, p2 state), beginning with the initial state (0,0).

time

p1

p2

CSS434 Time & Global States1 CSS434 Time and Global States Textbook Ch11 Professor: Munehiro Fukuda.

Documents

Transcript of CSS434 Time & Global States1 CSS434 Time and Global States Textbook Ch11 Professor: Munehiro Fukuda.