1

Principles of Reliable Distributed Systems

Lecture 3: Synchronous Uniform

Consensus

Spring 2006

Dr. Idit Keidar

2

Today’s Material

• Nancy Lynch, Distributed Algorithms, – Ch. 6

• Attiya and Welch, Distributed Computing,– Ch. 5

3

Reminder: State Machine Replication

a a ab b

c

4

Replica Coordination Requirements

• Agreement: replicas receive all client requests– What happens when a replica (server) fails?– What happens when a client fails?

• Order: replicas process requests in the same order

5

Uniform Atomic Broadcast

• Uniform Reliable Broadcast– Validity: if a correct process broadcasts m then all

correct processes eventually deliver m– Uniform Agreement: if some process delivers m then

all correct processes eventually deliver m– Integrity: m is delivered by a correct process at most

once, and only if it was previously broadcast

• Uniform Total Order– If two processes deliver both m and m’, they deliver

them in the same order

6

Today’s Problem: Uniform Consensus

Each process has an input, should on decide an output

• Uniform Agreement: all decisions are the same

• Validity: decision is input of one process

• Termination: eventually all correct processes decide

7

(Unifrom) Consensus versus (Uniform) Atomic Broadcast

• From Atomic Broadcast to Consensus

• From Consensus to Atomic Broadcast – Homework question

• From now on, we will focus mainly on consensus, and keep in mind that it suffices for Atomic Broadcast

8

Today’s Model

• Round-based synchronous

• Static set P = {p1, …, pn} of processes

• Crash failures

9

Round

Synchronous Model

• Synchronous rounds:

– send messages to any set of processes, – receive messages from this round, – do local processing (possibly decide, halt)

• If process pi crashes in a round, then any subset of the messages pi sends in this round can be lost

10

Round-Based Failstop Model

• If no message from pj is received, then pj is suspected• If pi fails in round r, then any subset of the messages pi

sends in r may arrive• If pi is suspected in round r, pi fails in round r or r-1

– no further messages from pi will arrive

round 1 round 2

p1

p2

p3

p1 crashes in round 2;

p2 receives p1’s round 2 message

p3 suspects p1 in round 2

11

t-Resilient Algorithm

• t is a threshold on the number of potential failures– the algorithm is correct as long as no more than t

processes fail

• In the following algorithm, 0 ≤ t < n• We denote by f the number of actual failures that

occur in a given run, 0 ≤ f ≤ t• We’d like t to be big (robust algorithm)

– but f will usually be small (failures are rare)

12

Notation

• P = {p1, …, pn} is the set of processes

• initi is pi’s initial value

• Local variables of pi are denoted: vi, Alivei

13

t-Resilient Failstop Uniform Consensus Algorithm

vi=initi; Alivei = P in every round 1 ≤ k ≤ t+2:

send vi to allreceive round k messagesfor all pj

if (received vj) then vi = min(vi, vj)otherwise pj is suspected

if ( (pj Alivei : received vj = vi) && !decided ) then decide vi.

for all pj if (suspect pj) then Alivei=Alivei {pj}

14

Proof: Validity

• Lemma: for every process pi, vi always includes the initial value initj of some process pj.

15

Proof: Uniform Agreement

• Lemma: – if exist value v, round r, and process pi s.t.

– all processes that are in Alivei at the beginning of round r send v in round r,

– then v is the only possible decision value from r onward.

16

Proof: Uniform Agreement (Cont’d)

• From the Lemma, we get that if some process decides v in round r, then v is the only possible decision value from r onward.

• Now look at the first round in which some process decides.

17

Proof: Termination

• After a round r in which no process fails, all processes have the same vi forever. – Because all receive the same messages in r,– By induction…

• Consider a run where f processes fail. Then for a correct process pi, Alivei changes in at most f rounds of this run.

• Thus, after at most f+2 rounds, there is a round in which Alivei does not change and all received values are the same.

18

How Long Does it Take?

• Early-deciding: in a run with f failures, decision is reached by the end of round f+2

• We will prove that this is optimal – for Uniform Consensus, but not for Consensus– as long as f < t-1

19

Deciding vs. Stopping (Halting)

• The algorithm is not early-stopping: – it continues running for t+2 rounds– even after reaching a decision

• Homework question: can you change the algorithm to be early-stopping?– stop (halt) after f+k rounds in runs with t≥f≥0 failures

for some constant k

20

Synchronous Authenticated Byzantine-Tolerant Consensus

21

Byzantine Faults

• Faulty process can behave arbitrarily, i.e., they don’t have to follow the protocol. E.g.,– can suffer benign failures – crash, timing;– can send bogus values in messages;– can send messages at the wrong time; – can send different messages to different

processes; etc.

• Captures software bugs, hacker intrusions.

22

Authenticated (Byzantine) Model

• Authentication: The receiver of a message can ascertain its origin;– an intruder cannot masquerade as someone else.

• Integrity: The receiver of a message can verify that it has not been modified in transit;– an intruder cannot substitute a false message for a

legitimate one.

• Nonrepudiation: A sender cannot falsely deny later that he sent a message.

23

Implementing Authentication

• Uses a Cryptographic Public Key Infrastructure (PKI).

• Each process has a well-know public key and a matching private key. Mp is message M signed by p’s private key.

– Only p can generate Mp .

– Every process can verify p’s signature on Mp using p’s public key.

24

Exploiting Authentication

• All messages are signed by their source.• Every receiver can verify that the message

was indeed sent by the source as is.• Signed messages can be forwarded as proof.

“I can prove that Idit said that I don’t have to submit this homework assignment” – Yossy does not have to submit homework assignment 2Idit

25

Consensus with Byzantine Failures

• Recall, we defined consensus as follows:– Agreement: correct processes’ decisions are

the same– Termination: eventually all correct processes

decide – Validity: decision is input of one process

• Problem?

26

Validity: Take II

• Strong unanimity: If the input of all the correct processes is v then no correct process decides a value other than v– When is this equivalent to the previous

definition?

• How resilient can an algorithm satisfying this property be?

27

Exponential Information Gathering (EIG) for t <n/2

send vi pi to allin every round 2 ≤ k ≤ t+1:

for every received message m:

if (m has k-1 different valid signatures) then send mpi to all the processes that did not sign it

Validi = {vjpj | all messages with t+1 valid signatures beginning with pj’s have same initial value vj }

decide on most common value in Validi (break ties)

28

Validity: Take III

• Weak unanimity: If the input of all the correct processes is v and no process fails then no correct process decides a value other than v

• Does this prevent a trivial solution?

29

Summary of Known Results

• Synchronous, Byzantine fault-tolerant, t-resilient consensus algorithms – – weak unanimity with authentication: iff t < n

• recitation

– strong unanimity with authentication: iff t < n/2– without authentication: iff t < n/3