CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda...
Transcript of CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda...
![Page 1: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/1.jpg)
CAP Theorem
![Page 2: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/2.jpg)
Definitions
Consistency: atomic, linearizable data items (each write appears to happen immediately across all nodes)
Availability: always get a response if your message goes through; no hanging
Partition tolerance: can lose messages (varying degrees)
![Page 3: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/3.jpg)
Group Partition
![Page 4: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/4.jpg)
Individual Partition
![Page 5: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/5.jpg)
(Some) Related Work
• Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected operation, sacrifice consistency
• The Bayou Project (e.g., SOSP 1995): Xerox PARC mobile device data synchronization, “anti-entropy” protocols
• “The dangers of replication and a solution” (Grey et. al, SIGMOD 1996): Lazy update propagation
![Page 6: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/6.jpg)
Brewer’s Work
• “Cluster Based Scalable Network Services” (SOSP 1997): Brewer and Inktomi, BASE principles
• “Harvest, Yield, and Scalable Tolerant Systems” (HotOS 1999): Brewer and Fox, actually describes Strong CAP Principle
![Page 7: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/7.jpg)
PODC 2000
![Page 8: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/8.jpg)
Do we believe CAP?
![Page 9: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/9.jpg)
Gilbert and Lynch
• Provide formal proof of CAP
• Use asynchronous network model
• No global clock
• Agents act on local state and messages only
![Page 10: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/10.jpg)
Theorem 1: It is impossible in the asynchronous network model to implement a read/write data object that guarantees the following properties:
• Availability
• Atomic consistency
in all fair executions (including those in which messages are lost)
![Page 11: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/11.jpg)
Theorem 1, EnglishYou can’t have C, A, and P if you have arbitrary message delays and message loss.
Makes sense: how can two groups communicate updates if they can’t communicate?
Key: availability requires that you return a value!
![Page 12: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/12.jpg)
Corollary 1.1: It is impossible in the asynchronous network model to implement a read/write data object that guarantees the following properties:
• Availability, in all fair executions,
• Atomic consistency, in fair executions in which no messages are lost
![Page 13: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/13.jpg)
Corollary 1.1, English
Too bad! In the asynchronous model, we can’t have C,A, and P even if we don’t have partitions!
Makes sense: impossible to determine if a message has been delayed or if it’s lost.
![Page 14: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/14.jpg)
Chicken little: the sky (cloud?) is falling!!!
Can we do anything useful?!?
![Page 15: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/15.jpg)
Of course;Use proof by example
![Page 16: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/16.jpg)
Recipe: C & P
def Handle_Request(socket):
close(socket);
return 0;
![Page 17: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/17.jpg)
Recipe: C & P
def Handle_Request(socket):
close(socket);
return 0;
never accept writes!!!
never return anything!!!
(never available, so no wrong answers)
![Page 18: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/18.jpg)
Recipe: C & A
Cake!
E.g., use a single master.
![Page 19: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/19.jpg)
Recipe: A & P
def Handle_Read(socket):
socket.write(init_value)
close(socket);
return 0;
def Handle_Write(socket):
socket.write(ACK);
//do nothing
close(socket);
return 0;
![Page 20: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/20.jpg)
Recipe: A & P
def Handle_Read(socket):
socket.write(init_value)
close(socket);
return 0;
always return initial value
(never consistent, trivially available)
def Handle_Write(socket):
socket.write(ACK);
//do nothing
close(socket);
return 0;
![Page 21: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/21.jpg)
...what if we bound network delays?
![Page 22: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/22.jpg)
...what if we bound network delays?
partial synchrony
![Page 23: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/23.jpg)
Theorem 2: It is impossible in the partially synchronous network model to
implement a read/write data object that guarantees the following properties:
• Availability
• Atomic consistency
in all executions (even those in which messages are lost).
![Page 24: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/24.jpg)
Theorem 2, English
Earthshaking: even with bounded message delays, if you lose messages arbitrarily, writes may not be propagated correctly and you’ll get stale data
Key: availability requires that you return a value!
![Page 25: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/25.jpg)
(Corollary 2.1): It is possible in the partially synchronous network model to implement a read/write data object that guarantees the following properties:
• Availability, in all fair executions,
• “Variable, sometimes atomic consistency”, in fair executions in which no messages are lost
![Page 26: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/26.jpg)
(Corollary 2.1), EnglishIn absence of message loss, if you don’t get an ack within 2*(max_msg_transit_time)+(time_spent_processing), then there was a partition!
Return consistent data in absence of partitions
Return inconsistent data with partitions, and detect this is happening
![Page 27: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/27.jpg)
(Quickly,) Delayed-t Consistency
• Weaker consistency form
• In a nutshell, partially order non-concurrent operations
• Use knowledge of timeouts to determine if messages are lost, and use sequence numbers and centralized node to define ordering
![Page 28: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/28.jpg)
Thoughts
• Do we need to have the formal proof in the paper?
• Formalism is nice to have...
• ...but it makes sense intuitively
![Page 29: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/29.jpg)
Thoughts
• w.r.t. good design, systems people always say “it depends”
• It’s nice to see a formalization of why “it depends”, and how “it depends” for once!
![Page 30: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/30.jpg)
Thoughts
• Lots of work making CAP tradeoffs implicitly before “CAP Theorem” announcement
• Was Brewer more perceptive than others?
• Would we still have BASE systems like Dynamo and Cassandra without formal CAP theorem?
• Who is the real Johnny Rotten here?
![Page 31: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/31.jpg)
Thoughts
• What about “Weak CAP Principle”? (HotOS 1999)
• “The stronger the guarantees made about any two of strong consistency, high availability, or resilience to partitions, the weaker the guarantees that can be made about the third.”
![Page 32: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/32.jpg)
Thoughts
• Daniel Abadi: PACELC
• “if there is a partition (P) how does the system tradeoff between availability and consistency (A and C); else (E) when the system is running as normal in the absence of partitions, how does the system tradeoff between latency (L) and consistency (C)?”
• http://dbmsmusings.blogspot.com/2010/04/problems-with-cap-and-yahoos-little.html
![Page 33: CAP Theorem - Peopleistoica/classes/cs294/... · 2011-09-21 · (Some) Related Work • Coda Project (e.g., IEEE Trans. on Computers 1990): CMU, high availability in disconnected](https://reader035.fdocuments.in/reader035/viewer/2022070709/5ebe3b55ab1ed31a9e2d1a75/html5/thumbnails/33.jpg)
End of Slides