High Throughput Byzantine Fault Tolerance
-
Upload
vance-morrison -
Category
Documents
-
view
44 -
download
4
description
Transcript of High Throughput Byzantine Fault Tolerance
High Throughput Byzantine Fault Tolerance
Ramakrishna Kotla, Mike DahlinLaboratory for Advanced Systems Research,
The University of Texas at Austin
April 19, 2023 Department of Computer Sciences, UT Austin 2
Summary of the talk
High throughput is achievable along with Byzantine fault tolerance
Contributions High Throughput BFT ArchitectureCBASE : Generic Prototype CBASE-FS : High throughput replicated NFS
April 19, 2023 Department of Computer Sciences, UT Austin 3
Outline
OverviewOverview
Architecture
Implementation
Evaluation
Conclusion
April 19, 2023 Department of Computer Sciences, UT Austin 4
Motivation
Large scale Internet services High Availability : 24 X 7 serviceHigh Reliability : CorrectnessHigh Security : Data integrity/ConfidentialityHigh Throughput : System load
Challenges : Byzantine failuresMalicious attacks
• http://www.cert.orgSoftware and operator errors
• ROC@USITS03Network and hardware failures
April 19, 2023 Department of Computer Sciences, UT Austin 6
BFT state machine replication
Byzantine Fault Tolerance ProtocolTolerates f Byzantine server failures using 3f+1
replicas Agreement : Order requests from clientsExecution stage : Execute requests
Provide high availability, reliability and securityPBFT, Farsite, Oceanstore [OSDI99, OSDI01, SOSP01, SOSP03]
Execution Execution Execution Execution
Agreement Agreement Agreement Agreement
Server Replicas
Clients
April 19, 2023 Department of Computer Sciences, UT Austin 7
BFT : Tradeoff throughput for fault tolerance ?
April 19, 2023 Department of Computer Sciences, UT Austin 8
Traditional BFT : Limitations
Fail to provide high throughput Does not scale with hardware resources and
application parallelism Reason
Uses Generalized State Machine Replication Correctness conditions:
• Agreement : Every non-faulty state machine replica receives every request
• Order : Every non-faulty state machine replica processes the requests in the same relative order
BFT State machine replication :Execute requests sequentially to ensure order
April 19, 2023 Department of Computer Sciences, UT Austin 9
High Throughput BFT : Idea
Modify Order without compromising consistency/safety Relaxed order : Every non-faulty replica executes
dependent requests in the same relative order Dependent requests : Two requests are dependent if
read set or write set of one intersects with write set of the other.
Requests that are not dependent can be concurrently executed
Exploit application parallelism to provide high throughput Commercial applications like web server, file systems,
databases have inherent data parallelism
April 19, 2023 Department of Computer Sciences, UT Austin 10
Outline
Overview
ArchitectureArchitecture
Implementation
Evaluation
Conclusion
April 19, 2023 Department of Computer Sciences, UT Austin 11
HT BFT : Architecture
Goals : Generic : Generic interface that exposes application parallelism Extensible : Easily extensible to support any application Modular : Support different fault models easily Reuse : Reuse existing agreement protocols
Parallelizer Parallelizer Parallelizer Parallelizer
Agreement Agreement Agreement Agreement
Server Replicas
Execution Execution ExecutionExecution
April 19, 2023 Department of Computer Sciences, UT Austin 12
Parallelizer
Application independent module Receives ordered requests from agreement Maintains/Updates dependency graph of
requests2 level dependency analysisConcurrency matrix
Schedules a request if it is not dependent on any outstanding requests (no outgoing edges at a request node)
Requests that are not dependent are concurrently executed
April 19, 2023 Department of Computer Sciences, UT Austin 13
Parallelizer : Concurrency Matrix
Definition/Figure : Square matrix rows/columns represent operations1 represents independent, 0 represents dependent
operations Exports application level parallelism
Statically defined Two matrices : Dependency also depends on
objectsRelated objectsUnrelated objects
Table Lookup Low overhead
April 19, 2023 Department of Computer Sciences, UT Austin 14
Parallelizer : Dependence Analysis
Parallelizer figure : agreement stage, input queue, dependency graph, multi thread execution stage
April 19, 2023 Department of Computer Sciences, UT Austin 15
Advantages/Limitations
Advantages :Supports high throughput applicationsSimple : Minimal/No changes to client/agreement
protocol/application Flexible : Supports different fault models easily
Limitation :Concurrency matrix requires inner workings of
application Conservative rules ensures correctness at the
expense of performance Incrementally refine the rules to gain performance
April 19, 2023 Department of Computer Sciences, UT Austin 16
Outline
Overview
Architecture
ImplementationImplementation
Evaluation
Conclusion
April 19, 2023 Department of Computer Sciences, UT Austin 17
System Model
Asynchronous systemNodes operate at arbitrarily different speedsNetwork may delay, drop or deliver messages out
of orderAssumption : Bounded fair links
Fault Model : Byzantine FaultsFaulty nodes may behave arbitrarily : crash,
lose/alter data, send incorrect messages Adversary : Strong adversary
Can coordinate faulty nodes in arbitrarily bad waysAssumption : Computationally limited
April 19, 2023 Department of Computer Sciences, UT Austin 18
CBASE : Concurrent BASE
Uses unmodified PBFT agreement protocol [OSDI 1999]
Built upon BASE library [SOSP 2001] Agreement stage : Single thread Execution stage : Multithreaded Parallelizer : Producer/Consumer queue Figure ??
April 19, 2023 Department of Computer Sciences, UT Austin 19
Parallelizer : Interface
Parallelizer.insert() Parallelizer.next_request() Parallelizer.sync()
April 19, 2023 Department of Computer Sciences, UT Austin 20
CBASE-FS : BFT NFS
Figure Brief description of NFS concurrency matrix rules
Related objects : Same NFS handleRules are conservativeRefer paper for more details
April 19, 2023 Department of Computer Sciences, UT Austin 21
Outline
Overview
Architecture
Implementation
EvaluationEvaluation
Conclusion
April 19, 2023 Department of Computer Sciences, UT Austin 22
Evaluation
With 4 server replicas that tolerate 1 Byzantine failure
Replicas running on different uniprocessor machine 933 MHz P3, 256 MB Ram
5 Client machines Dedicated network with 100MB ethernet hub OS : Redhat Linux 7.2 with NFS 2.0 Assumption : No correlated failures due to OS.
April 19, 2023 Department of Computer Sciences, UT Austin 23
Microbenchmark : Overhead
BASE versus CBASE
April 19, 2023 Department of Computer Sciences, UT Austin 24
Microbenchmark : Scalability
Scalability with hardware resources Scalability with application level parallelism
April 19, 2023 Department of Computer Sciences, UT Austin 25
Microbenchmark : CBASE-FS/BASE-FS/NFS
Latency versus Throughput with no sleep Latency versus Throughput with 20 ms sleep Iozone results summary
April 19, 2023 Department of Computer Sciences, UT Austin 27
Conclusions
Commercial applications have parallelism High throughput BFT provides a simple/flexible
solution to achieve high throughput