Webinar: Eventual Consistency != Hopeful Consistency
-
Upload
datastax -
Category
Technology
-
view
125 -
download
5
description
Transcript of Webinar: Eventual Consistency != Hopeful Consistency
![Page 1: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/1.jpg)
Eventual Consistency != Hopeful Consistency
Embracing Optimistic Design in the Persistence Layer
![Page 2: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/2.jpg)
Who am I?Christos KalantzisNetflix Inc.Engineering Manager – Cloud Database Engineering
@[email protected]/in/christoskalantzis
![Page 3: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/3.jpg)
C* Replication & Consistency Recap• C* is eventually Consistent– That means it WILL get there…eventually
• Eventually is not:– A day from now– A minute from now– A second from now
• In most cases it is milliseconds
![Page 4: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/4.jpg)
• C* has tunable consistency– All
– Quorum
– One
C* Replication & Consistency Recap
![Page 5: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/5.jpg)
Remember When...?• In early 2000s, Read was scaled out by adding
replication Slaves to a Master DB.• Writes went to a single Master, Reads went to
Slaves
![Page 6: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/6.jpg)
Remember When…?• Slaves could lose transactions
![Page 7: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/7.jpg)
Remember When…?• No “Repair” function
![Page 8: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/8.jpg)
Remember When…?• We trusted it• We still trust it
![Page 9: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/9.jpg)
C* Consistency Concerns• “I want high consistency in my Reads/Writes just like I had
in my RDBMS setup”– You never really had it.
• “I want my DB to catch integrity issues”– We’ve been turning FK off for years!
• See Rails, Grails & other MVC frameworks
• Can I trust that C* will replicate my data when writing at CL 1”
![Page 10: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/10.jpg)
Netflix Experiment• Created a multi-datacenter C* 1.1.7 cluster of 48 nodes in each
DC• Put load on C* Cluster
– 100K total operations per second (50 K in each DC)• Wrote 1,000,000 records at CL1 in one DC• Read same 1,000,000 records at CL1 in other DC• ALL records were read successfully• You can trust it!
![Page 11: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/11.jpg)
Netflix Experiment
![Page 12: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/12.jpg)
Optimistic vs. Pessimistic Design• Pessimistic Design– Design with high consistency, you punish your
users 99.9% of the time• Higher consistency = higher latency• Diminished user experience
![Page 13: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/13.jpg)
Optimistic vs. Pessimistic Design• Optimistic Design– Trust your data store
• Know your business and your application– Always ask yourself, is it really that important?
• Handle edge cases through contingency plans
![Page 14: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/14.jpg)
Low Consistency Example 1Retail
![Page 15: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/15.jpg)
Low Consistency Example 2Banking
![Page 16: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/16.jpg)
Hurdles Faced• Engineers are stubborn!
– 1+1=2 .. not eventually 2• Middle management is scared
– It’s a feat you convinced them to use C*– Now you got to convince them to accept low consistency?
• Engaging Product team to implement some type of contingency plans
![Page 17: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/17.jpg)
How to Overcome those Hurdles• Prove it through a POC• Show them the benefits of an improved user
experience– Its all about the user
• You may be working for the wrong company– We’re hiring!– jobs.netflix.com
![Page 18: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/18.jpg)
Q&A
![Page 19: Webinar: Eventual Consistency != Hopeful Consistency](https://reader035.fdocuments.in/reader035/viewer/2022062511/54c665b14a79592d268b4583/html5/thumbnails/19.jpg)
Thank you!