Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Replication in Databases and Distributed SystemsCourse: CS655
Rabiet Louis
Colorado State University
Thursday 5 December 2013
1 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
1 Eager and lazy replications
2 Databases and distributed systems
3 Streaming application
4 CRDTTreedoc: Cooperative editing
5 Hybrid Replication
6 Thoughts
7 CCL
2 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Plan
1 Eager and lazy replications
2 Databases and distributed systems
3 Streaming application
4 CRDTTreedoc: Cooperative editing
5 Hybrid Replication
6 Thoughts
7 CCL
3 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Analysis of replication in DB [GH96]
Hypothesis
All the nodes are suppose to have a copy of the data (nopartition, no scaling up)
Eager and lazy replication are not appropriate solutions [nextslides].
Eager: Synchronization then commit.Lazy: Asynchronous propagation.
Fixed number of objects in the database.
Fixed number of nodes (but may be unavailable).
No delay in communications.
4 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Analysis of replication in DB [GH96]
Hypothesis
All the nodes are suppose to have a copy of the data (nopartition, no scaling up)
Eager and lazy replication are not appropriate solutions [nextslides].
Eager: Synchronization then commit.Lazy: Asynchronous propagation.
Fixed number of objects in the database.
Fixed number of nodes (but may be unavailable).
No delay in communications.
4 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Analysis of replication in DB [GH96]
Hypothesis
All the nodes are suppose to have a copy of the data (nopartition, no scaling up)
Eager and lazy replication are not appropriate solutions [nextslides].
Eager: Synchronization then commit.
Lazy: Asynchronous propagation.
Fixed number of objects in the database.
Fixed number of nodes (but may be unavailable).
No delay in communications.
4 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Analysis of replication in DB [GH96]
Hypothesis
All the nodes are suppose to have a copy of the data (nopartition, no scaling up)
Eager and lazy replication are not appropriate solutions [nextslides].
Eager: Synchronization then commit.Lazy: Asynchronous propagation.
Fixed number of objects in the database.
Fixed number of nodes (but may be unavailable).
No delay in communications.
4 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Analysis of replication in DB [GH96]
Hypothesis
All the nodes are suppose to have a copy of the data (nopartition, no scaling up)
Eager and lazy replication are not appropriate solutions [nextslides].
Eager: Synchronization then commit.Lazy: Asynchronous propagation.
Fixed number of objects in the database.
Fixed number of nodes (but may be unavailable).
No delay in communications.
4 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Analysis of replication in DB [GH96]
Hypothesis
All the nodes are suppose to have a copy of the data (nopartition, no scaling up)
Eager and lazy replication are not appropriate solutions [nextslides].
Eager: Synchronization then commit.Lazy: Asynchronous propagation.
Fixed number of objects in the database.
Fixed number of nodes (but may be unavailable).
No delay in communications.
4 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Analysis of replication in DB [GH96]
Hypothesis
All the nodes are suppose to have a copy of the data (nopartition, no scaling up)
Eager and lazy replication are not appropriate solutions [nextslides].
Eager: Synchronization then commit.Lazy: Asynchronous propagation.
Fixed number of objects in the database.
Fixed number of nodes (but may be unavailable).
No delay in communications.
4 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Eager Replication
Just one node.
Node Deadlock Rate ≈ Trans Deadlock Rate ∗ Nb Trans
Prob Deadlock
Lifetime OneTrans
Trans Per sec ∗ Nb Actions ∗ OneAction Time
Prob Deadlock ≈ (Prob Waiting)2 ∗ Nb Trans
5 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Eager Replication
Just one node.
Node Deadlock Rate ≈ Trans Deadlock Rate ∗ Nb Trans
Prob Deadlock
Lifetime OneTrans
Trans Per sec ∗ Nb Actions ∗ OneAction Time
Prob Deadlock ≈ (Prob Waiting)2 ∗ Nb Trans
5 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Eager Replication
Just one node.
Node Deadlock Rate ≈ Trans Deadlock Rate ∗ Nb Trans
Prob Deadlock
Lifetime OneTrans
Trans Per sec ∗ Nb Actions ∗ OneAction Time
Prob Deadlock ≈ (Prob Waiting)2 ∗ Nb Trans
5 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Eager Replication
Just one node.
Node Deadlock Rate ≈ Trans Deadlock Rate ∗ Nb Trans
Prob Deadlock
Lifetime OneTrans
Trans Per sec ∗ Nb Actions ∗ OneAction Time
Prob Deadlock ≈ (Prob Waiting)2 ∗ Nb Trans
5 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Eager Replication
Generalization.
Nb Trans’ = Trans Per sec’ ∗ Lifetime OneTrans’
Trans Per sec ∗ Nb Nodes
Nb Actions ∗ OneAction Time ∗ Nb Nodes
Scalability issue
Nb Trans’ in Nb Nodes2 order
6 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Eager Replication
Generalization.
Nb Trans’ = Trans Per sec’ ∗ Lifetime OneTrans’
Trans Per sec ∗ Nb Nodes
Nb Actions ∗ OneAction Time ∗ Nb Nodes
Scalability issue
Nb Trans’ in Nb Nodes2 order
6 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Eager Replication
Generalization.
Nb Trans’ = Trans Per sec’ ∗ Lifetime OneTrans’
Trans Per sec ∗ Nb Nodes
Nb Actions ∗ OneAction Time ∗ Nb Nodes
Scalability issue
Nb Trans’ in Nb Nodes2 order
6 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Eager Replication
Generalization.
Nb Trans’ = Trans Per sec’ ∗ Lifetime OneTrans’
Trans Per sec ∗ Nb Nodes
Nb Actions ∗ OneAction Time ∗ Nb Nodes
Scalability issue
Nb Trans’ in Nb Nodes2 order
6 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Eager Replication
Generalization.
Eager Deadlock =Prob Deadlock Eager
Lifetime OneTrans’∗ Nb Trans’
Prob Deadlock Eager ≈ (Prob Waiting)2 ∗ Nb Trans’
Scalability issue
Eager Deadlock in Nb Nodes3 order
7 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Eager Replication
Generalization.
Eager Deadlock =Prob Deadlock Eager
Lifetime OneTrans’∗ Nb Trans’
Prob Deadlock Eager ≈ (Prob Waiting)2 ∗ Nb Trans’
Scalability issue
Eager Deadlock in Nb Nodes3 order
7 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Eager Replication
Generalization.
Eager Deadlock =Prob Deadlock Eager
Lifetime OneTrans’∗ Nb Trans’
Prob Deadlock Eager ≈ (Prob Waiting)2 ∗ Nb Trans’
Scalability issue
Eager Deadlock in Nb Nodes3 order
7 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Lazy Group Replication
Motivation
scalable.
support disconnection of nodes.
Reminder
Any node can update data (no master) = Group Replication.
Reconciliation of conflict (timestamps are used) is needed (nodeadlock in lazy group schema).
8 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Lazy Group Replication
Motivation
scalable.
support disconnection of nodes.
Reminder
Any node can update data (no master) = Group Replication.
Reconciliation of conflict (timestamps are used) is needed (nodeadlock in lazy group schema).
8 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Lazy Group Replication
Motivation
scalable.
support disconnection of nodes.
Reminder
Any node can update data (no master) = Group Replication.
Reconciliation of conflict (timestamps are used) is needed (nodeadlock in lazy group schema).
8 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Lazy Group Replication
Motivation
scalable.
support disconnection of nodes.
Reminder
Any node can update data (no master) = Group Replication.
Reconciliation of conflict (timestamps are used) is needed (nodeadlock in lazy group schema).
8 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Lazy Group Replication
Motivation
scalable.
support disconnection of nodes.
Reminder
Any node can update data (no master) = Group Replication.
Reconciliation of conflict (timestamps are used) is needed (nodeadlock in lazy group schema).
8 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Lazy Group Replication
Lazy Reconcialiation Rate = Prob Collision ∗ Nb Nodes
Disconnection Time
Inbound Updates ∗ Outbound Updates
DB Size
(in o(Nb Nodes))
in o(Nb Nodes)
9 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Lazy Group Replication
Lazy Reconcialiation Rate = Prob Collision ∗ Nb Nodes
Disconnection Time
Inbound Updates ∗ Outbound Updates
DB Size
(in o(Nb Nodes))
in o(Nb Nodes)
9 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Lazy Master
A master is assigned to each object.
Conflict are in o(Nb Nodes2).
No support for disconnection.
10 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Lazy Master
A master is assigned to each object.
Conflict are in o(Nb Nodes2).
No support for disconnection.
10 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Lazy Master
A master is assigned to each object.
Conflict are in o(Nb Nodes2).
No support for disconnection.
10 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Two-Tier Replication
Slight modification of the lazy master replication schema.
Restriction: at most a mobile node is involved in thetransaction.
A mobile (in the case where it is the master) while connect toa base node (that contains a replica of the data) and willupdate his content with this new data.
11 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Two-Tier Replication
Slight modification of the lazy master replication schema.
Restriction: at most a mobile node is involved in thetransaction.
A mobile (in the case where it is the master) while connect toa base node (that contains a replica of the data) and willupdate his content with this new data.
11 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Two-Tier Replication
Slight modification of the lazy master replication schema.
Restriction: at most a mobile node is involved in thetransaction.
A mobile (in the case where it is the master) while connect toa base node (that contains a replica of the data) and willupdate his content with this new data.
11 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Plan
1 Eager and lazy replications
2 Databases and distributed systems
3 Streaming application
4 CRDTTreedoc: Cooperative editing
5 Hybrid Replication
6 Thoughts
7 CCL
12 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Motivation
Databases and distributed systems use replication.
One for Performance. One for failure resilience.
Idea is to compare them.
13 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Motivation
Databases and distributed systems use replication.
One for Performance. One for failure resilience.
Idea is to compare them.
13 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Motivation
Databases and distributed systems use replication.
One for Performance. One for failure resilience.
Idea is to compare them.
13 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
5 Phases in replication
Request.
Server Coordination.
Execution.
Agreement Coordination.
Response.
14 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
5 Phases in replication
Request.
Server Coordination.
Execution.
Agreement Coordination.
Response.
14 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
5 Phases in replication
Request.
Server Coordination.
Execution.
Agreement Coordination.
Response.
14 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
5 Phases in replication
Request.
Server Coordination.
Execution.
Agreement Coordination.
Response.
14 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
5 Phases in replication
Request.
Server Coordination.
Execution.
Agreement Coordination.
Response.
14 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Distributed System replication
Active replication.
Passive replication.
Semi-active replication.
15 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Distributed System replication
Active replication.
Passive replication.
Semi-active replication.
15 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Distributed System replication
Active replication.
Passive replication.
Semi-active replication.
15 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Database replication One operation.
Eager primary copy.
Eager update everywherewith distributed locking.
Eager update everywherebased on atomic broadcast.
Lazy update everywhere.
16 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Database replication One operation.
Eager primary copy.
Eager update everywherewith distributed locking.
Eager update everywherebased on atomic broadcast.
Lazy update everywhere.
16 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Database replication One operation.
Eager primary copy.
Eager update everywherewith distributed locking.
Eager update everywherebased on atomic broadcast.
Lazy update everywhere.
16 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Database replication One operation.
Eager primary copy.
Eager update everywherewith distributed locking.
Eager update everywherebased on atomic broadcast.
Lazy update everywhere.
16 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Database replication Several operations.
Eager primary copy.
Eager update everywherewith distributed locking.
Certification replication.
17 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Database replication Several operations.
Eager primary copy.
Eager update everywherewith distributed locking.
Certification replication.
17 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Database replication Several operations.
Eager primary copy.
Eager update everywherewith distributed locking.
Certification replication.
17 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Plan
1 Eager and lazy replications
2 Databases and distributed systems
3 Streaming application
4 CRDTTreedoc: Cooperative editing
5 Hybrid Replication
6 Thoughts
7 CCL
18 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Application [hHBR+04]
Interested in recovery.
Precise > Rollback > Gap.
Streams of data.
Pairs of primary/secondary.
19 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Application [hHBR+04]
Interested in recovery.
Precise > Rollback > Gap.
Streams of data.
Pairs of primary/secondary.
19 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Application [hHBR+04]
Interested in recovery.
Precise > Rollback > Gap.
Streams of data.
Pairs of primary/secondary.
19 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Application [hHBR+04]
Interested in recovery.
Precise > Rollback > Gap.
Streams of data.
Pairs of primary/secondary.
19 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Gap Recovery
In case of failure or partial communication, just ignore thequeries.
Called amnesia.
No recovery.
20 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Gap Recovery
In case of failure or partial communication, just ignore thequeries.
Called amnesia.
No recovery.
20 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Gap Recovery
In case of failure or partial communication, just ignore thequeries.
Called amnesia.
No recovery.
20 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Passive standby
Sending the input queues, operators and output queues to thesecondary node.
To avoid suspension during check-pointing: sweep line.
Strongest property for recovery.
21 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Passive standby
Sending the input queues, operators and output queues to thesecondary node.
To avoid suspension during check-pointing: sweep line.
Strongest property for recovery.
21 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Passive standby
Sending the input queues, operators and output queues to thesecondary node.
To avoid suspension during check-pointing: sweep line.
Strongest property for recovery.
21 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Upstream backup
Upstream nodes act as backup for downstream nodes.
22 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Upstream backup
Upstream nodes act as backup for downstream nodes.
22 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Upstream backup
Upstream nodes act as backup for downstream nodes.
22 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Active standby
Same as passive standby except that upstream secondarynodes receive the tuples and are processing it.
Because of non deterministic operators, input must berecorded to keep the relation input,output.
23 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Active standby
Same as passive standby except that upstream secondarynodes receive the tuples and are processing it.
Because of non deterministic operators, input must berecorded to keep the relation input,output.
23 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Some extensions to make precise recovery
Passive standby: verification with downstream nodes.
Active standby: distribute information to replay operations.
Upstream Backup: using the watermarks.
24 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Some extensions to make precise recovery
Passive standby: verification with downstream nodes.
Active standby: distribute information to replay operations.
Upstream Backup: using the watermarks.
24 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Some extensions to make precise recovery
Passive standby: verification with downstream nodes.
Active standby: distribute information to replay operations.
Upstream Backup: using the watermarks.
24 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Experimentation
recovery time bandwith overhead processing overhead
Amnesia 0 0 0
Passive K + Qp f1(1/M,C) f2(1/M,C)
Upstream K + Q ′p f3(1/M, c) f4(1/M, ops, paths)
Active ε 100% + f3(1/M, c) 100% + 2 ∗ f4(1/M, ops, paths)
25 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Plan
1 Eager and lazy replications
2 Databases and distributed systems
3 Streaming application
4 CRDTTreedoc: Cooperative editing
5 Hybrid Replication
6 Thoughts
7 CCL
26 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
CRDT
Commutativity =⇒ Convergence simpler
27 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
CRDT
SEC (Strong eventual Consistency).
Eventual delivery ∀i , j , f ∈ ci =⇒ 3f ∈ cjStrong convergence ∀i , j , ci ∈ cj =⇒ si ≡ sjTermination
Without any conflict resolution.
n − 1 failures possibles.
28 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
CRDT
SEC (Strong eventual Consistency).
Eventual delivery ∀i , j , f ∈ ci =⇒ 3f ∈ cj
Strong convergence ∀i , j , ci ∈ cj =⇒ si ≡ sjTermination
Without any conflict resolution.
n − 1 failures possibles.
28 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
CRDT
SEC (Strong eventual Consistency).
Eventual delivery ∀i , j , f ∈ ci =⇒ 3f ∈ cjStrong convergence ∀i , j , ci ∈ cj =⇒ si ≡ sj
Termination
Without any conflict resolution.
n − 1 failures possibles.
28 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
CRDT
SEC (Strong eventual Consistency).
Eventual delivery ∀i , j , f ∈ ci =⇒ 3f ∈ cjStrong convergence ∀i , j , ci ∈ cj =⇒ si ≡ sjTermination
Without any conflict resolution.
n − 1 failures possibles.
28 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
CRDT
SEC (Strong eventual Consistency).
Eventual delivery ∀i , j , f ∈ ci =⇒ 3f ∈ cjStrong convergence ∀i , j , ci ∈ cj =⇒ si ≡ sjTermination
Without any conflict resolution.
n − 1 failures possibles.
28 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
CRDT
SEC (Strong eventual Consistency).
Eventual delivery ∀i , j , f ∈ ci =⇒ 3f ∈ cjStrong convergence ∀i , j , ci ∈ cj =⇒ si ≡ sjTermination
Without any conflict resolution.
n − 1 failures possibles.
28 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Examples of CDRT
Vectors counters.
(Nn, [0, ..., 0],≥n, value, inc,maxn)
if pi can only update the i-th element in the vector ≡ vector-clock
29 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Sufficient condition
Eventual delivery
Termination
Monotonic semi-lattice
=⇒ SEC
30 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Treedoc: Cooperative editing
Treedoc [PMSL09]
c
b
a
0
0
e
d
0
f
1
1
31 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Treedoc: Cooperative editing
Treedoc [PMSL09]
c
b
a
0
0
e
d
0
f
1
1
dC
dB
dA
dE
dD dF
32 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Treedoc: Cooperative editing
Treedoc [PMSL09]
33 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Plan
1 Eager and lazy replications
2 Databases and distributed systems
3 Streaming application
4 CRDTTreedoc: Cooperative editing
5 Hybrid Replication
6 Thoughts
7 CCL
34 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Hybrid Quorum [CML+06]
Use a quick quorum if no contention.
BFT [CL02] if contention.
Combine quorum and agreement based state-machinereplication.
35 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Hybrid Quorum [CML+06]
Use a quick quorum if no contention.
BFT [CL02] if contention.
Combine quorum and agreement based state-machinereplication.
35 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Hybrid Quorum [CML+06]
Use a quick quorum if no contention.
BFT [CL02] if contention.
Combine quorum and agreement based state-machinereplication.
35 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Normal Case
Write operations = 2 phases.
Read operations = 1 phase
3f + 1 replica.
Phase Write1 = timestamp t obtained using a quorum.
Phase Write2 = Execution (2f + 1 must agree to execute theoperation at t).
36 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Normal Case
Write operations = 2 phases.
Read operations = 1 phase
3f + 1 replica.
Phase Write1 = timestamp t obtained using a quorum.
Phase Write2 = Execution (2f + 1 must agree to execute theoperation at t).
36 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Normal Case
Write operations = 2 phases.
Read operations = 1 phase
3f + 1 replica.
Phase Write1 = timestamp t obtained using a quorum.
Phase Write2 = Execution (2f + 1 must agree to execute theoperation at t).
36 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Normal Case
Write operations = 2 phases.
Read operations = 1 phase
3f + 1 replica.
Phase Write1 = timestamp t obtained using a quorum.
Phase Write2 = Execution (2f + 1 must agree to execute theoperation at t).
36 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Normal Case
Write operations = 2 phases.
Read operations = 1 phase
3f + 1 replica.
Phase Write1 = timestamp t obtained using a quorum.
Phase Write2 = Execution (2f + 1 must agree to execute theoperation at t).
36 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Contention
if Write2 has been completed, it is still assigned.
if Write1 has obtained t then the operation will be executedat t ′ ≥ t
37 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Contention
if Write2 has been completed, it is still assigned.
if Write1 has obtained t then the operation will be executedat t ′ ≥ t
37 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Plan
1 Eager and lazy replications
2 Databases and distributed systems
3 Streaming application
4 CRDTTreedoc: Cooperative editing
5 Hybrid Replication
6 Thoughts
7 CCL
38 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
CRDT
3 Applicable only on commutative operations.
3 Not many use cases?
39 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
CRDT
3 Applicable only on commutative operations.
3 Not many use cases?
39 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Streaming
3 Number of replica = 2.
Checkpointing.
Load Balancing between the two nodes.
Several loops (image processing).
Path exploration.
40 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Streaming
3 Number of replica = 2.
Checkpointing.
Load Balancing between the two nodes.
Several loops (image processing).
Path exploration.
40 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Streaming
3 Number of replica = 2.
Checkpointing.
Load Balancing between the two nodes.
Several loops (image processing).
Path exploration.
40 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Streaming
3 Number of replica = 2.
Checkpointing.
Load Balancing between the two nodes.
Several loops (image processing).
Path exploration.
40 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Streaming
3 Number of replica = 2.
Checkpointing.
Load Balancing between the two nodes.
Several loops (image processing).
Path exploration.
40 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Plan
1 Eager and lazy replications
2 Databases and distributed systems
3 Streaming application
4 CRDTTreedoc: Cooperative editing
5 Hybrid Replication
6 Thoughts
7 CCL
41 / 43
Eager and lazy replications Databases and distributed systems Streaming application CRDT Hybrid Replication Thoughts CCL
Conclusion
Thank You for your attention !
42 / 43
Bibliography
Miguel Castro and Barbara Liskov.
Practical byzantine fault tolerance and proactive recovery.ACM Trans. Comput. Syst., 20(4):398–461, November 2002.
James Cowling, Daniel Myers, Barbara Liskov, Rodrigo Rodrigues, and Liuba Shrira.
Hq replication: A hybrid quorum protocol for byzantine fault tolerance.In Proceedings of the 7th Symposium on Operating Systems Design and Implementation, OSDI ’06, pages177–190, Berkeley, CA, USA, 2006. USENIX Association.
Jim Gray and Pat Helland.
The dangers of replication and a solution.In In Proceedings of the 1996 ACM SIGMOD International Conference on Management of Data, pages173–182, 1996.
Jeong hyon Hwang, Magdalena Balazinska, Er Rasin, Michael Stonebraker, and Stan Zdonik.
High-availability algorithms for distributed stream processing.Technical report, 2004.
Nuno Preguica, Joan Manuel Marques, Marc Shapiro, and Mihai Letia.
A commutative replicated data type for cooperative editing.In Proceedings of the 2009 29th IEEE International Conference on Distributed Computing Systems, ICDCS’09, pages 395–403, Washington, DC, USA, 2009. IEEE Computer Society.
Marc Shapiro, Nuno Preguica, Carlos Baquero, and Marek Zawirski.
Conflict-free replicated data types.In Proceedings of the 13th International Conference on Stabilization, Safety, and Security of DistributedSystems, SSS’11, pages 386–400, Berlin, Heidelberg, 2011. Springer-Verlag.
Matthias Wiesmann, Fernando Pedone, Andre Schiper, Betina Kemme, and Gustavo Alonso.
Understanding replication in databases and distributed systems.pages 264–274, 2000.
Top Related