Concepts and Values Mário Tomás Catarina Rodrigues André Rodrigues 20/05/09.
Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart &...
-
date post
22-Dec-2015 -
Category
Documents
-
view
219 -
download
6
Transcript of Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart &...
![Page 1: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/1.jpg)
Byzantine Techniques II
Presenter: Georgios Piliouras
Partly based on slides by
Justin W. Hart & Rodrigo Rodrigues
![Page 2: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/2.jpg)
Papers
• Practical Byzantine Fault Tolerance. Miguel Castro et. al. (OSDI 1999)
• BAR Fault Tolerance for Cooperative Services. Amitanand S. Aiyer, et. al. (SOSP 2005)
![Page 3: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/3.jpg)
Motivation
• Computer systems provide crucial services
server
client
![Page 4: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/4.jpg)
Problem
• Computer systems provide crucial services
• Computer systems fail– natural disasters
– hardware failures
– software errors
– malicious attacks
Need highly-available services
client
server
![Page 5: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/5.jpg)
Replication
unreplicated service
client
server
![Page 6: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/6.jpg)
Replication
replicated service
client
serverreplicas
unreplicated service
client
server
Replication algorithm: • masks a fraction of faulty replicas• high availability if replicas fail “independently”
![Page 7: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/7.jpg)
Assumptions are a Problem
• Replication algorithms make assumptions:– behavior of faulty processes
– synchrony
– bound on number of faults
• Service fails if assumptions are invalid
![Page 8: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/8.jpg)
Assumptions are a Problem
• Replication algorithms make assumptions:– behavior of faulty processes
– synchrony
– bound on number of faults
• Service fails if assumptions are invalid– attacker will work to invalidate assumptions
Most replication algorithms assume too much
![Page 9: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/9.jpg)
Contributions
• Practical replication algorithm:– weak assumptions tolerates attacks– good performance
• Implementation– BFT: a generic replication toolkit– BFS: a replicated file system
• Performance evaluation
BFS is only 3% slower than a standard file system
![Page 10: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/10.jpg)
Talk Overview
• Problem
• Assumptions
• Algorithm
• Implementation
• Performance
• Conclusions
![Page 11: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/11.jpg)
Bad Assumption: Benign Faults
• Traditional replication assumes:– replicas fail by stopping or omitting steps
![Page 12: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/12.jpg)
Bad Assumption: Benign Faults
• Traditional replication assumes:– replicas fail by stopping or omitting steps
• Invalid with malicious attacks:– compromised replica may behave arbitrarily– single fault may compromise service– decreased resiliency to malicious attacks
client
serverreplicas
attacker replacesreplica’s code
![Page 13: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/13.jpg)
BFT Tolerates Byzantine Faults
• Byzantine fault tolerance: – no assumptions about faulty behavior
• Tolerates successful attacks– service available when hacker controls replicas
client
serverreplicas
attacker replacesreplica’s code
![Page 14: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/14.jpg)
Byzantine-Faulty Clients
• Bad assumption: client faults are benign– clients easier to compromise than replicas
![Page 15: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/15.jpg)
Byzantine-Faulty Clients
• Bad assumption: client faults are benign– clients easier to compromise than replicas
• BFT tolerates Byzantine-faulty clients:– access control– narrow interfaces– enforce invariants server
replicas
attacker replacesclient’s code
Support for complex service operations is important
![Page 16: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/16.jpg)
Bad Assumption: Synchrony
• Synchrony known bounds on:
– delays between steps
– message delays
• Invalid with denial-of-service attacks:– bad replies due to increased delays
• Assumed by most Byzantine fault tolerance
![Page 17: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/17.jpg)
Asynchrony
• No bounds on delays• Problem: replication is impossible
![Page 18: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/18.jpg)
Asynchrony
• No bounds on delays• Problem: replication is impossible
Solution in BFT:• provide safety without synchrony
– guarantees no bad replies
![Page 19: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/19.jpg)
Asynchrony
• No bounds on delays• Problem: replication is impossible
Solution in BFT:• provide safety without synchrony
– guarantees no bad replies
• assume eventual time bounds for liveness– may not reply with active denial-of-service attack– will reply when denial-of-service attack ends
![Page 20: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/20.jpg)
Talk Overview
• Problem
• Assumptions
• Algorithm
• Implementation
• Performance
• Conclusions
![Page 21: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/21.jpg)
Algorithm Properties• Arbitrary replicated service
– complex operations – mutable shared state
• Properties (safety and liveness):– system behaves as correct centralized service– clients eventually receive replies to requests
• Assumptions:– 3f+1 replicas to tolerate f Byzantine faults (optimal)
– strong cryptography– only for liveness: eventual time bounds
clients
replicas
![Page 22: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/22.jpg)
State machine replication:– deterministic replicas start in same state– replicas execute same requests in same order– correct replicas produce identical replies
Algorithm
replicasclient
![Page 23: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/23.jpg)
State machine replication:– deterministic replicas start in same state– replicas execute same requests in same order– correct replicas produce identical replies
Algorithm
replicasclient
Hard: ensure requests execute in same order
![Page 24: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/24.jpg)
Ordering Requests
Primary-Backup:• View designates the primary replica
• Primary picks ordering• Backups ensure primary behaves correctly
– certify correct ordering– trigger view changes to replace faulty primary
view
replicasclientprimary backups
![Page 25: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/25.jpg)
Rough Overview of Algorithm
• A client sends a request for a service to the primary
replicasclientprimary backups
![Page 26: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/26.jpg)
Rough Overview of Algorithm
• A client sends a request for a service to the primary
• The primary mulicasts the request to the backups
replicasclientprimary backups
![Page 27: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/27.jpg)
Rough Overview of Algorithm
• A client sends a request for a service to the primary
• The primary mulicasts the request to the backups• Replicas execute request and sent a reply to the
client
replicasclientprimary backups
![Page 28: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/28.jpg)
Rough Overview of Algorithm• A client sends a request for a service to the
primary• The primary mulicasts the request to the backups• Replicas execute request and sent a reply to the
client• The client waits for f+1 replies from different
replicas with the same result; this is the result of the operation
view
replicasclientprimary backups
f+1 matching replies
![Page 29: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/29.jpg)
Quorums and Certificates
3f+1 replicas
quorums have at least 2f+1 replicas
quorum A quorum B
quorums intersect in at least one correct replica
• Certificate set with messages from a quorum
• Algorithm steps are justified by certificates
![Page 30: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/30.jpg)
Algorithm Components
• Normal case operation
• View changes
• Garbage collection
• Recovery
All have to be designed to work together
![Page 31: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/31.jpg)
Normal Case Operation
• Three phase algorithm:– pre-prepare picks order of requests– prepare ensures order within views– commit ensures order across views
• Replicas remember messages in log
• Messages are authenticated • denotes a message sent by kk
![Page 32: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/32.jpg)
Pre-prepare Phase
request : m
assign sequence number n to request m in view v
primary = replica 0
replica 1
replica 2
replica 3fail
multicast PRE-PREPARE,v,n,m0
backups accept pre-prepare if:• in view v• never accepted pre-prepare for v,n with different request
![Page 33: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/33.jpg)
Prepare Phase
mpre-prepare prepare
replica 0
replica 1
replica 2
replica 3 fail
multicast PREPARE,v,n,D(m),11
digest of m
accepted PRE-PREPARE,v,n,m0
all collect pre-prepare and 2f matching prepares
P-certificate(m,v,n)
![Page 34: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/34.jpg)
Order Within View
If it were false:
replicas
quorum forP-certificate(m’,v,n)
quorum forP-certificate(m,v,n)
one correct replica in common m = m’
No P-certificates with the same view and sequence number and different requests
![Page 35: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/35.jpg)
Commit Phase
Request m executed after:• having C-certificate(m,v,n) • executing requests with sequence number less than n
replica has P-certificate(m,v,n)
mpre-prepare prepare
replica 0
replica 1
replica 2
replica 3fail
commit
multicast COMMIT,v,n,D(m),22
all collect 2f+1 matching commits
C-certificate(m,v,n)
replies
![Page 36: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/36.jpg)
View Changes
• Provide liveness when primary fails: – timeouts trigger view changes – select new primary ( view number mod 3f+1)
![Page 37: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/37.jpg)
View Changes
• Provide liveness when primary fails: – timeouts trigger view changes – select new primary ( view number mod 3f+1)
• But also need to: – preserve safety– ensure replicas are in the same view long enough– prevent denial-of-service attacks
![Page 38: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/38.jpg)
View Change Protocol
replica 0 = primary v
replica 1= primary v+1
replica 2
replica 3
fail
send P-certificates: VIEW-CHANGE,v+1,P,22
primary collects VC-messages in X: NEW-VIEW,v+1,X,O1
pre-prepares messages for v+1 view in O with the same sequence number
backups multicast prepare messages for pre-prepares in O
2f VC messages
![Page 39: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/39.jpg)
View Change Safety
• Intuition: if replica has C-certificate(m,v,n) then
any quorum Qquorum forC-certificate(m,v,n)
correct replica in Q has P-certificate(m,v,n)
Goal: No C-certificates with the same sequence number and different requests
![Page 40: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/40.jpg)
Garbage CollectionTruncate log with certificate: • periodically checkpoint state (K) • multicast CHECKPOINT,n,D(checkpoint),i• all collect 2f+1 checkpoint messages
send checkpoint in view-changes
i
Log
h H=h+2K
discard messages and checkpoints
reject messages
sequence numbers
S-certificate(h,checkpoint)
![Page 41: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/41.jpg)
Formal Correctness Proofs
• Complete safety proof with I/O automata– invariants – simulation relations
•
Partial liveness proof with timed I/O automata
– invariants
![Page 42: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/42.jpg)
Communication Optimizations
• Digest replies: send only one reply to client with result
![Page 43: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/43.jpg)
Communication Optimizations
• Digest replies: send only one reply to client with result
• Optimistic execution: execute prepared requests
Read-write operations execute in two round-trips
client 2f+1 replies
![Page 44: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/44.jpg)
Communication Optimizations
• Digest replies: send only one reply to client with result
• Optimistic execution: execute prepared requests
• Read-only operations: executed in current state
Read-only operations execute in one round-trip
client
Read-write operations execute in two round-trips
client 2f+1 replies
2f+1 replies
![Page 45: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/45.jpg)
Talk Overview
• Problem
• Assumptions
• Algorithm
• Implementation
• Performance
• Conclusions
![Page 46: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/46.jpg)
BFS: A Byzantine-Fault-Tolerant NFS
No synchronous writes – stability through replication
andrew benchmark
kernel NFS client
relay
replicationlibrary
snfsdreplication
library
kernel VM
snfsdreplication
library
kernel VM
replica 0
replica n
![Page 47: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/47.jpg)
Talk Overview
• Problem
• Assumptions
• Algorithm
• Implementation
• Performance
• Conclusions
![Page 48: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/48.jpg)
Andrew Benchmark
0
10
20
30
40
50
60
70
BFS BFS-nr
• BFS-nr is exactly like BFS but without replication• 30 times worse with digital signatures
Configuration
• 1 client, 4 replicas• Alpha 21064, 133 MHz• Ethernet 10 Mbit/s
Ela
ps
ed
tim
e (
sec
on
ds
)
![Page 49: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/49.jpg)
BFS is Practical
0
10
20
30
40
50
60
70
BFS NFS
• NFS is the Digital Unix NFS V2 implementation
Configuration
• 1 client, 4 replicas• Alpha 21064, 133 MHz• Ethernet 10 Mbit/s• Andrew benchmark
Ela
ps
ed
tim
e (
sec
on
ds
)
![Page 50: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/50.jpg)
BFS is Practical 7 Years Later
050
100150200250300350400450500
BFS BFS-nr NFS
• NFS is the Linux 2.2.12 NFS V2 implementation
Configuration
• 1 client, 4 replicas• Pentium III, 600MHz• Ethernet 100 Mbit/s• 100x Andrew benchmark
Ela
ps
ed
tim
e (
sec
on
ds
)
![Page 51: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/51.jpg)
Conclusions
Byzantine fault tolerance is practical:– Good performance
– Weak assumptions improved resiliency
![Page 52: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/52.jpg)
What happens if we go MAD?
![Page 53: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/53.jpg)
What happens if we go MAD?
• Several useful cooperative services span Multiple Administrative Domains.– Internet routing– File distribution– Cooperative backup e.t.c.
• Dealing only with Byzantine behaviors is not enough.
![Page 54: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/54.jpg)
Why?
• Nodes are under control of multiple administrators
![Page 55: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/55.jpg)
Why?
• Nodes are under control of multiple administrators
• Broken – Byzantine behaviors. – Misconfigured, or configured with malicious
intent.
![Page 56: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/56.jpg)
Why?
• Nodes are under control of multiple administrators
• Broken – Byzantine behaviors. – Misconfigured, or configured with malicious
intent.
• Selfish – Rational behaviors– Alter the protocol to increase local utility
![Page 57: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/57.jpg)
Talk Overview
• Problem
• Model
• 3 Level Architecture
• Performance
• Conclusions
![Page 58: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/58.jpg)
It is time to raise the BAR
![Page 59: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/59.jpg)
It is time to raise the BAR
• Byzantine– Behaving arbitrarily or maliciously
• Altruistic– Execute the proposed program, whether it
benefits them or not
• Rational– Deviate from the proposed program for
purposes of local benefit
![Page 60: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/60.jpg)
Protocols
• Incentive-Compatible Byzantine Fault Tolerant (IC-BFT)– It is in the best interest of rational nodes to follow
the protocol exactly
![Page 61: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/61.jpg)
Protocols
• Incentive-Compatible Byzantine Fault Tolerant (IC-BFT)– It is in the best interest of rational nodes to follow
the protocol exactly
• Byzantine Altruistic Rational Tolerant (BART)– Guarantees a set of safety and liveliness
properties despite the presence of rational nodes
• IC-BFT BART
![Page 62: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/62.jpg)
General idea
• Extend/Modify the Practical Byzantine Fault Tolerance Model in a way that combats the negative effects of rational (greedy) behavior.
• We will achieve that by using game-theoretic tools.
+
![Page 63: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/63.jpg)
A taste of Nash Equilibrium
X_X,X_X
-100,-100
+1,-1Go Straight
-1,+10, 0Swerve
Go StraightSwerve
![Page 64: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/64.jpg)
Naughty nodes are punished
• Nodes require access to a state machine in order to complete their objectives
• Protocol contains methods for punishing rational nodes, including denying them access to the state machine
![Page 65: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/65.jpg)
Talk Overview
• Problem
• BAR Model
• 3 Level Architecture
• Performance
• Conclusions
![Page 66: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/66.jpg)
Three-Level Architecture
• Layered design• simplifies analysis/construction of systems• isolates classes of misbehavior at
appropriate levels of abstraction
![Page 67: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/67.jpg)
Level 1 – Basic Primitives
• Goals:– Provide IC-BFT versions of key
abstractions – Ensure long-term benefit to participants– Limit non-determinism– Mitigate the efffects of residual non-
determinism– Enforce predictable communication
patterns
![Page 68: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/68.jpg)
Level 1 – Basic Primitives
• Goals:– Provide IC-BFT versions of key
abstractions – Ensure long-term benefit to participants– Limit non-determinism– Mitigate the efffects of residual non-
determinism– Enforce predictable communication
patterns
![Page 69: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/69.jpg)
Level 1 – Basic Primitives
BART-RSM based on PBFT
Differences: use TRB instead of consensus
3f+2 nodes required for f faulty
![Page 70: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/70.jpg)
Level 1 – Basic Primitives
• Goals:– Provide IC-BFT versions of key
abstractions – Ensure long-term benefit to participants– Limit non-determinism– Mitigate the efffects of residual non-
determinism– Enforce predictable communication
patterns
![Page 71: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/71.jpg)
Level 1 – Basic Primitives
• The RSM rotates the leadership role to participants.
• Participants want to stay in the system in order to control the RSM and complete their protocols
• Ultimately, incentives stem from the higher level service
![Page 72: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/72.jpg)
Level 1 – Basic Primitives
• Goals:– Provide IC-BFT versions of key
abstractions – Ensure long-term benefit to participants– Limit non-determinism– Mitigate the efffects of residual non-
determinism– Enforce predictable communication
patterns
![Page 73: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/73.jpg)
Level 1 – Basic Primitives
• Self interested nodes could hide behind non-determinism to shirk work.– Tit-for-Tat policy– Communicate proofs of misconducts, leads to global
punishment
• Use Terminating Reliable Broadcast, rather than consensus.– In TRB, only the sender can propose a value– Other nodes can only adopt this value, or choose a
default value
![Page 74: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/74.jpg)
Level 1 – Basic Primitives
• Goals:– Provide IC-BFT versions of key
abstractions – Ensure long-term benefit to participants– Limit non-determinism– Mitigate the efffects of residual non-
determinism– Enforce predictable communication
patterns
![Page 75: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/75.jpg)
Level 1 – Basic Primitives
• Balance costs– No incentive to make the wrong choice
• Encourage timeliness– By allowing nodes to judge unilaterally
whether other nodes’ messages are late and inflict sanctions to them (Penance)
![Page 76: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/76.jpg)
Level 1 – Basic Primitives
• Goals:– Provide IC-BFT versions of key
abstractions – Ensure long-term benefit to participants– Limit non-determinism– Mitigate the efffects of residual non-
determinism– Enforce predictable communication
patterns
![Page 77: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/77.jpg)
Level 1 – Basic Primitives
• Nodes have to have participated at every step in order to have the opportunity to issue a command
• Message queues
x y
x x x xyyy
I am waiting message from x
![Page 78: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/78.jpg)
Level 1 – Basic Primitives
• Nodes have to have participated at every step in order to have the opportunity to issue a command
• Message queues
x y
x x x xyyy
message
![Page 79: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/79.jpg)
Level 1 – Basic Primitives
• Nodes have to have participated at every step in order to have the opportunity to issue a command
• Message queues
x y
xx xyyy
I am waiting message from y
y
![Page 80: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/80.jpg)
Theorems
• Theorem 1: The TRB protocol satisfies Termination, Agreement, Integrity and Non-Triviality
• Theorem 2: No node has a unilateral incentive to deviate from the protocol
![Page 81: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/81.jpg)
Level 2
• State machine replication is sufficient to support a backup service, but the overhead is unacceptable– 100 participants… 100 MB backed up… 10
GB of drive space
• Assign work to individual nodes, using arithmetic codes to provide low-overhead fault-tolerant storage
![Page 82: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/82.jpg)
Guaranteed Response
• Direct communication is insufficient when nodes can behave rationally
• We introduce a “witness” that overhears the conversation
• This eliminates ambiguity
• Messages are routed through this intermediary
![Page 83: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/83.jpg)
Guaranteed Response
![Page 84: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/84.jpg)
Guaranteed Response
• Node A sends a request to Node B through the witness
• The witness stores the request, and enters RequestReceived state
• Node B sends a response to Node A through the witness
• The witness stores the response, and enters ResponseReceived
![Page 85: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/85.jpg)
Guaranteed Response
• Deviation from this protocol will cause the witness to either notice the timeout from Node B or lying on the part of Node A
![Page 86: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/86.jpg)
Optimization through Credible Threats
![Page 87: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/87.jpg)
Optimization through Credible Threats
• Returns to game theory• Protocol is optimized so nodes can
communicate directly. Add a fast path• If recipient does not respond, nodes proceed
to the unoptimized case• Analogous to a game of chicken
![Page 88: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/88.jpg)
Periodic Work Protocol
• Witness checks that periodic tasks, such as system maintenance are performed
• It is expected that, with a certain frequency, each node in the system will perform such a task
• Failure to perform one will generate a POM from the witness
![Page 89: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/89.jpg)
Authoritative Time Service
• Maintains authoritative time
• Binds messages sent to that time
• Guaranteed response protocol relies on this for generating NoResponses
![Page 90: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/90.jpg)
Authoritative Time Service
• Each submission to the state machine contains the timestamp of the proposer
• Timestamp is taken to be the maximum of the median of timestamps of the previous f+1 decisions
• If “no decision” is decided, then the timestamp is the previous authoritative time
![Page 91: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/91.jpg)
Level 3 BAR-B
• BAR-B is a cooperative backup system
• Three operations– Store– Retrieve– Audit
![Page 92: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/92.jpg)
Storage
• Nodes break files up into chunks
• Chunks are encrypted
• Chunks are stored on remote nodes
• Remote nodes send signed receipts and store StoreInfos
![Page 93: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/93.jpg)
Retrieval
• A node storing a chunk can respond to a request for a chunk with– The chunk– A demonstration that the chunk’s lease has
expired– A more recent StoreInfo
![Page 94: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/94.jpg)
Auditing
• Receipts constitute audit records
• Nodes will exchange receipts in order to verify compliance with storage quotas
![Page 95: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/95.jpg)
Talk Overview
• Problem
• BAR Model
• 3 Level Architecture
• Performance
• Conclusions
![Page 96: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/96.jpg)
Evaluation
• Performance is inferior to protocols that do note make these guarantees, but acceptable (?)
![Page 97: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/97.jpg)
Impact of additional nodes
![Page 98: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/98.jpg)
Impact of rotating leadership
![Page 99: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/99.jpg)
Impact of fast path optimization
![Page 100: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/100.jpg)
Talk Overview
• Problem
• BAR Model
• 3 Level Architecture
• Performance
• Conclusions
![Page 101: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/101.jpg)
Conclusions
• More useful as a proof of concept but certainly explores a very interesting common ground between systems and game theory as a way of exploring the performance of real-life systems.
![Page 102: Byzantine Techniques II Presenter: Georgios Piliouras Partly based on slides by Justin W. Hart & Rodrigo Rodrigues.](https://reader036.fdocuments.in/reader036/viewer/2022062715/56649d7b5503460f94a5f41e/html5/thumbnails/102.jpg)
Conclusions
• More useful as a proof of concept but certainly explores a very interesting common ground between systems and game theory as a way of exploring the performance of real-life systems.
• CS 614 is over