Distributed systems in practice, in theory (ScaleConf Colombia)
Distributed systems in practice, in theory
-
Upload
aysylu-greenberg -
Category
Software
-
view
1.236 -
download
1
Transcript of Distributed systems in practice, in theory
![Page 1: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/1.jpg)
Confidential + ProprietaryConfidential + Proprietary
Distributed Systems in Practice, in TheoryAysylu GreenbergMarch 8th, 2016
![Page 2: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/2.jpg)
Confidential + Proprietary
![Page 3: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/3.jpg)
Confidential + Proprietary
![Page 4: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/4.jpg)
Confidential + Proprietary
Aysylu Greenberg
@aysylu22
![Page 5: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/5.jpg)
Confidential + Proprietary
Towards Distributed Build System
![Page 6: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/6.jpg)
Confidential + Proprietary
Towards Distributed Build System
![Page 7: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/7.jpg)
Confidential + Proprietary
http://opencv.org/
Towards Distributed Build System
![Page 8: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/8.jpg)
Confidential + Proprietary
http://opencv.org/
Towards Distributed Build System
![Page 9: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/9.jpg)
Confidential + Proprietary
http://opencv.org/
Towards Distributed Build System
![Page 10: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/10.jpg)
Confidential + Proprietary
http://opencv.org/
Towards Distributed Build System
![Page 11: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/11.jpg)
Confidential + Proprietary
http://opencv.org/
Towards Distributed Build System
![Page 12: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/12.jpg)
Confidential + Proprietary
http://opencv.org/
Towards Distributed Build System
![Page 13: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/13.jpg)
Confidential + Proprietary
http://opencv.org/
Towards Distributed Build System
![Page 14: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/14.jpg)
Confidential + Proprietary
http://opencv.org/
Towards Distributed Build System
![Page 15: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/15.jpg)
Confidential + Proprietary
http://opencv.org/
Towards Distributed Build System
![Page 16: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/16.jpg)
Confidential + Proprietary
http://opencv.org/
Towards Distributed Build System
![Page 17: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/17.jpg)
Confidential + Proprietary
http://opencv.org/
Towards Distributed Build System
![Page 18: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/18.jpg)
Confidential + Proprietary
http://opencv.org/
Towards Distributed Build System
![Page 19: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/19.jpg)
Confidential + Proprietary
http://opencv.org/
Towards Distributed Build System
![Page 20: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/20.jpg)
Confidential + Proprietary
http://opencv.org/
Towards Distributed Build System
![Page 21: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/21.jpg)
Confidential + Proprietary
http://opencv.org/
Towards Distributed Build System
![Page 22: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/22.jpg)
Confidential + Proprietary
![Page 23: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/23.jpg)
Confidential + Proprietary
Papers We Love SF
![Page 24: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/24.jpg)
Confidential + Proprietary
Aysylu Greenberg
@aysylu22
![Page 25: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/25.jpg)
Confidential + Proprietarymeetup.com/Papers-We-Love-London
![Page 26: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/26.jpg)
Confidential + Proprietary
Today
● Staged Event-Driven Architecture
![Page 27: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/27.jpg)
Confidential + Proprietary
Today
● Staged Event-Driven Architecture● Leases
![Page 28: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/28.jpg)
Confidential + Proprietary
Today
● Staged Event-Driven Architecture● Leases● Inaccurate Computations
![Page 29: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/29.jpg)
Confidential + ProprietaryConfidential + Proprietary
Computer Science ResearchInDistributed Systems Industry
![Page 30: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/30.jpg)
Confidential + Proprietary
Operating systems research
![Page 31: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/31.jpg)
Confidential + Proprietary
Operating systems research
![Page 32: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/32.jpg)
Confidential + Proprietary
Operating systems research
Concurrency
![Page 33: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/33.jpg)
Confidential + Proprietary
Operating systems research
Concurrency
Concurrency primitives: mutex & semaphore
![Page 34: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/34.jpg)
Confidential + Proprietary
Operating systems research
Concurrency
Concurrency primitives: mutex & semaphore
Processes execute at different speeds
![Page 35: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/35.jpg)
Confidential + Proprietary
Time in distributed systems
![Page 36: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/36.jpg)
Confidential + Proprietary
Time in distributed systems
![Page 37: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/37.jpg)
Confidential + Proprietary
Time in distributed systems
Pipelining
![Page 38: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/38.jpg)
Confidential + Proprietary
Time in distributed systems
Pipelining
![Page 39: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/39.jpg)
Confidential + Proprietary
Internet
![Page 40: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/40.jpg)
Confidential + Proprietary
Internet
Distributed consensus
![Page 41: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/41.jpg)
Confidential + Proprietary
Internet
Distributed consensus
![Page 42: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/42.jpg)
Confidential + Proprietary
Internet
Distributed consensus
![Page 43: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/43.jpg)
Confidential + Proprietary
Internet
Distributed consensus
Paxos
![Page 44: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/44.jpg)
Confidential + Proprietary
Reconsider large systems
![Page 45: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/45.jpg)
Confidential + Proprietary
Reconsider large systems
Platform as a service
![Page 46: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/46.jpg)
Confidential + Proprietary
CS Research is Timeless
Inform decisions
Mitigate technical risk
![Page 47: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/47.jpg)
Confidential + ProprietaryConfidential + Proprietary
Staged Event Driven Architecture&Deep Pipelines
2001
![Page 48: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/48.jpg)
Confidential + Proprietary
Hardware Pipelines
![Page 49: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/49.jpg)
Confidential + Proprietary
Data Pipelines
https://en.wikipedia.org/wiki/Graphics_pipeline
![Page 50: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/50.jpg)
Confidential + Proprietary
Search Indexing Pipelines
![Page 51: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/51.jpg)
Confidential + Proprietary
Search Indexing Pipelines
![Page 52: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/52.jpg)
Confidential + Proprietary
Search Indexing Pipelines
![Page 53: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/53.jpg)
Confidential + Proprietary
Search Indexing Pipelines
![Page 54: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/54.jpg)
Confidential + Proprietary
Search Indexing Pipelines
![Page 55: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/55.jpg)
Confidential + Proprietary
Search Indexing Pipelines
![Page 56: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/56.jpg)
Confidential + Proprietary
Search Indexing Pipelines
![Page 57: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/57.jpg)
Confidential + Proprietary
Search Indexing Pipelines
![Page 58: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/58.jpg)
Confidential + Proprietary
Search Indexing Pipelines
![Page 59: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/59.jpg)
Confidential + Proprietary
Search Indexing Pipelines
![Page 60: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/60.jpg)
Confidential + Proprietary
Search Indexing Pipelines
![Page 61: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/61.jpg)
Confidential + Proprietary
![Page 62: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/62.jpg)
Confidential + Proprietary
Staged Event Driven Architecture
![Page 63: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/63.jpg)
Confidential + Proprietary
Single-machine pipeline
generalizes to distributed pipelines
Staged Event Driven Architecture
![Page 64: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/64.jpg)
Confidential + Proprietary
Staged Event Driven Architecture
● Dynamic resource controllers
![Page 65: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/65.jpg)
Confidential + Proprietary
Staged Event Driven Architecture
● Dynamic resource controllers
+-
![Page 66: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/66.jpg)
Confidential + Proprietary
Staged Event Driven Architecture
● Dynamic resource controllers○ automatic tuning
+-
![Page 67: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/67.jpg)
Confidential + Proprietary
Staged Event Driven Architecture
● Dynamic resource controllers○ automatic tuning
■ thread pool sizing+
-
![Page 68: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/68.jpg)
Confidential + Proprietary
Staged Event Driven Architecture
● Dynamic resource controllers○ automatic tuning
■ thread pool sizing■ event batching +
-
![Page 69: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/69.jpg)
Confidential + Proprietary
+-
Staged Event Driven Architecture
● Dynamic resource controllers○ automatic tuning
■ thread pool sizing■ event batching
○ load shedding via queue
![Page 70: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/70.jpg)
Confidential + Proprietary
+-
Staged Event Driven Architecture
● Dynamic resource controllers○ automatic tuning
■ thread pool sizing■ event batching
○ load shedding via queue■ backpressure
![Page 71: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/71.jpg)
Confidential + Proprietary
+-
Staged Event Driven Architecture
● Dynamic resource controllers○ automatic tuning
■ thread pool sizing■ event batching
○ load shedding via queue■ backpressure■ threshold, filter, reorder, aggregate
![Page 72: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/72.jpg)
Confidential + ProprietaryConfidential + Proprietary
Leasesas Heart Beat in Distributed Systems 1989
![Page 73: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/73.jpg)
Confidential + Proprietary
![Page 74: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/74.jpg)
Confidential + Proprietary
Leases
● Distributed locking
![Page 75: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/75.jpg)
Confidential + Proprietary
Leases
● Distributed locking● Lease term tradeoffs
○ short
![Page 76: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/76.jpg)
Confidential + Proprietary
Leases
● Distributed locking● Lease term tradeoffs
○ short■ delay from client and server failures
minimized■ reduced false write-sharing
![Page 77: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/77.jpg)
Confidential + Proprietary
Leases
● Distributed locking● Lease term tradeoffs
○ short vs long
![Page 78: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/78.jpg)
Confidential + Proprietary
Leases
● Distributed locking● Lease term tradeoffs
○ short vs long■ more efficient for frequently accessed
data■ minimized lease extension overhead on
server and client
![Page 79: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/79.jpg)
Confidential + Proprietary
Leases
● Distributed locking● Lease term tradeoffs
○ short vs long● Use of leases in modern applications
○ Leader election TTL (in etcd)
![Page 80: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/80.jpg)
Confidential + Proprietary
Leases
● Distributed locking● Lease term tradeoffs
○ short vs long● Use of leases in modern applications
○ Leader election TTL (in etcd)○ Liveness detection
![Page 81: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/81.jpg)
Confidential + Proprietary
![Page 82: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/82.jpg)
Confidential + Proprietary
Do the build, s'il vous plaît!
Build System
Dita Roque-Gourary
![Page 83: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/83.jpg)
Confidential + Proprietary
Do the build, s'il vous plaît!
Build System
OK/D'Accord!
Dita Roque-Gourary
![Page 84: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/84.jpg)
Confidential + Proprietary
Do the build, s'il vous plaît!
Build System
OK/D'Accord!
Waiting for the results, merci.
Dita Roque-Gourary
![Page 85: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/85.jpg)
Confidential + Proprietary
Do the build, s'il vous plaît!
Build System
OK/D'Accord!
Waiting for the results, merci.
Build is in progress
Dita Roque-Gourary
![Page 86: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/86.jpg)
Confidential + Proprietary
Do the build, s'il vous plaît!
Build System
OK/D'Accord!
Waiting for the results, merci.
Build is in progress
Dita Roque-Gourary
Waiting for the results, merci.
![Page 87: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/87.jpg)
Confidential + Proprietary
Do the build, s'il vous plaît!
Build System
OK/D'Accord!
Waiting for the results, merci.
Build is in progress
Build is finished/fini.Dita Roque-Gourary
Waiting for the results, merci.
![Page 88: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/88.jpg)
Confidential + Proprietary
Leases in Build System
![Page 89: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/89.jpg)
Confidential + ProprietaryConfidential + Proprietary
Inaccurate Computations&Serving Search Results
![Page 90: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/90.jpg)
Confidential + Proprietary
From Accurate to"Good Enough"
![Page 91: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/91.jpg)
Confidential + Proprietary
![Page 92: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/92.jpg)
Confidential + Proprietary
![Page 93: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/93.jpg)
Confidential + Proprietary
![Page 94: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/94.jpg)
Confidential + Proprietary
![Page 95: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/95.jpg)
Confidential + Proprietary
![Page 96: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/96.jpg)
Confidential + Proprietary
Inaccuracy for Resilience
1. Task decomposition
![Page 97: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/97.jpg)
Confidential + Proprietary
Inaccuracy for Resilience
1. Task decomposition2. Baseline for correctness
![Page 98: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/98.jpg)
Confidential + Proprietary
Inaccuracy for Resilience
1. Task decomposition2. Baseline for correctness3. Criticality Testing
![Page 99: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/99.jpg)
Confidential + Proprietary
Inaccuracy for Resilience
1. Task decomposition2. Baseline for correctness3. Criticality Testing4. Distortion and timing models
![Page 100: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/100.jpg)
Confidential + ProprietaryJeff Dean "Building Software Systems at Google and Lessons Learned", Stanford, 2010
![Page 101: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/101.jpg)
Confidential + ProprietaryJeff Dean "Building Software Systems at Google and Lessons Learned", Stanford, 2010
![Page 102: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/102.jpg)
Confidential + Proprietary
![Page 103: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/103.jpg)
Confidential + Proprietary
![Page 104: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/104.jpg)
Confidential + Proprietary
Robust & scaleable pipelines
![Page 105: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/105.jpg)
Confidential + Proprietary
Robust & scaleable pipelinesLeases for sharing & heartbeat
![Page 106: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/106.jpg)
Confidential + Proprietary
Robust & scaleable pipelinesLeases for sharing & heartbeat
Trade off inaccuracy for resilience & performance
![Page 107: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/107.jpg)
Confidential + Proprietary
Robust, scaleable pipelinesLeases for sharing & heartbeat
Trade off inaccurate for resilient
CS research is timeless:
use it to mitigate risk
![Page 108: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/108.jpg)
Confidential + Proprietary
ThanksInes SombraDavid GreenbergAlex HutchesonScott ZawalskiKaran Parikh
![Page 109: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/109.jpg)
Confidential + Proprietary
References● T. Wurthinger, C. Wimmer et al. "One VM to Rule Them All"● M. Rinard "Probabilistic Accuracy Bounds for Fault-Tolerant
Computations that Discard Tasks"● F. Corbato, M. Daggett, R. Daley "An Experimental Time-
Sharing System"● E. Dijkstra "Cooperating Sequential Processes"● L. Lamport "Time, Clocks, and the Ordering of Events in a
Distributed System"
![Page 110: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/110.jpg)
Confidential + Proprietary
References● B. Oki, B. Liskov "Viewstamped Replication: A New Primary Copy
Method to Support Highly-Available Distributed Systems"● L. Lamport "The Part-Time Parliament"● M. Welsh, D. Culler, E. Brewer "SEDA: An Architecture for Well-
Conditioned, Scalable Internet Services"● C. Gray, D. Cheriton "Leases: An Efficient Fault-Tolerant
Mechanism for Distributed File Cache Consistency"● S. Agarwal, B. Mozafari et al. "BlinkDB: Queries with Bounded
Errors and Bounded Response Times on Very Large Data"
![Page 111: Distributed systems in practice, in theory](https://reader033.fdocuments.in/reader033/viewer/2022042907/587fac7d1a28ab107e8b49db/html5/thumbnails/111.jpg)
Confidential + Proprietary
Should I read papers?
YES