Raft assignments - Princeton University Computer Science · •Assignment 3: an example in...
Transcript of Raft assignments - Princeton University Computer Science · •Assignment 3: an example in...
![Page 1: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/1.jpg)
Raft assignments
COS 418: Distributed SystemsPrecept 9
Themis Melissaris and Daniel Suo
![Page 2: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/2.jpg)
Agenda
• Generalobservations• Assignment3:anexampleindesigninganimplementation
• Assignment4:expandingontheexample• Assignment5:avoidingcommonpitfalls
2
![Page 3: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/3.jpg)
Onincrementalassignments
• Wehearyou:nothavingsolutionstoearlierassignmentswhenlaterassignmentsdependonthemishard
• Beatingthedeadhorse:– Thisreflectstherealitymoreoftenthannotinsoftwareengineering
– Thisisalsoaforcingmechanismtoreallyunderstandadistributedsystemandhowtomakegooddesignchoices
3
![Page 4: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/4.jpg)
#1reasonforstruggle:repeatinglogic• Someexamples
– Usingmultiplestatevariablesforonestate– HandlingheartbeatandAppendEntries aredifferent(morerelevantforA4)
– StartnewelectionfromCandidateandFolloweraredifferent
– Resettingtimers
4
![Page 5: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/5.jpg)
Wedon’twantourcodetobe
• Rigid:difficulttochange;needtotouchmanyplacestomakesimplechanges
• Fragile:changesbreaksysteminunexpectedways
• Immobile:hardtoreuselogic/code
Theseadjectivescausedpeoplealotofpain!We’llrevisitthroughoutthisprecept
5
![Page 6: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/6.jpg)
Assignment3
6
![Page 7: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/7.jpg)
WhyarewegoingoverA3again?
• Shouldbereview• Wespendalotoftimeinclassdescribingsystems
• Youspendalotoftimeinassignmentsimplementingsystems
• Aquick(andsimpler)exampleofhowwegofromasystemdescriptiontosystemimplementation
• Therearemanypossibleimplementations!7
![Page 8: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/8.jpg)
Reasoningaboutstate
• Assignment3asksustoimplementastatemachine(i.e.,elections)
• Eachraftserverhasa‘state’(Follower,Candidate,Leader)
• Raftservercanchangestateaccordingtocertainrules
8
![Page 9: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/9.jpg)
Reasoningaboutstate
9
![Page 10: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/10.jpg)
Reasoningaboutstate
• Whatadditionalstatemusteachserverhold?– currentTerm– votedFor
• WewillalsoassumeeachserverhasaTimerobject,butthereareotherimplementationstohandletimeouts(e.g.,timeoutloops)
10
![Page 11: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/11.jpg)
Reasoningabouttransitions(Follower)• Howcanwebecomeafollower?
– Whenwefirststart– WhenwereceiveanRPCfromaserverwithahighercurrentTerm
• Followerscanonlybecomecandidates(directly,anyway)
11
![Page 12: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/12.jpg)
Reasoningabouttransitions(Candidate)• Howcanwebecomeacandidate?
– Whenweareafollowerandhaven’treceivedaheartbeatfromaleaderwithintheelectiontimeout
– Whenweareacandidateandhaven’tbeenvotedleaderorheardfromaleaderwithintheelectiontimeout
• CandidatescanbecomeanyofFollower,Candidate,orLeader
12
![Page 13: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/13.jpg)
Reasoningabouttransitions(Leader)• Howcanwebecomealeader?
– Whenwearecandidateandifwereceivevotesfrommajorityofserverswithinelectiontimeout
• LeadercanbecomeFollower– Ifweseeaserverwithahigherterm– Typicallyhappensafterwedieorthereisapartition
13
![Page 14: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/14.jpg)
Whendowechangestate?• Thereisatimeout
– Follower->Candidate– Candidate->Candidate
• WereceiveanRPC– Leader->Follower– Candidate->Follower
• WehandlearesponsetoanRPC– Leader->Follower– Candidate->Leader– Candidate->Follower
14
![Page 15: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/15.jpg)
Ourcodeshouldreflect!• Timingout
– resetTimer• Createatimerifthereisn’tone(i.e.,whenweMake)andstartgoroutine tocall
handleTimer wheneverthereistimeout• Settimeouttoheartbeatintervalifweareleader,torandomizedelectionintervalif
wearenot(note,thesamewhetherweareFollowerorCandidate)
– handleTimer• Ifleader,callsendAppendEntries• Otherwise,becomecandidate(notethislogicisthesameifweareFolloweror
Candidate)
• ReceivinganRPC– RequestVote:specifiedinpaper(don’tneedtoimplementthewhole
thing)– AppendEntries:specifiedinpaper(don’tneedtoimplementthe
wholething)15
![Page 16: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/16.jpg)
Ourcodeshouldreflect!• HandlingRPCresponses
– handleRequestVoteResponse:specifiedinpaper(don’tneedtoimplementthewholething)
– handleAppendEntriesResponse:specifiedinpaper(don’tneedtoimplementthewholething)
• SendingRPCs– sendRequestVote:sendRequestVote inseparategoroutine toeach
server;callhandleRequestVoteResponse onresponse– sendAppendEntries:sendAppendEntries inseparategoroutine to
eachserver;callhandleAppendEntriesResponse onresponse
16
![Page 17: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/17.jpg)
Somedetails(assumingarchitectureinpreviousslides)• Resettingtimer
– Whenwestart– Wheneverwehandletimeout– Wheneverwechangestate
• Locking/unlocking(whendowemodifystate?)– Whenwehandletimeout– WhenwereceiveanRPC– WhenwehandleasingleRPCresponse
• ResettingvotedFor tonull(or-1)– WhenwebecomefollowerexceptinAppendEntries
17
![Page 18: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/18.jpg)
Assignment4
18
![Page 19: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/19.jpg)
High-leveloverview
• Assumearchitecturefromearlierslides• PartI
– Modifyallfunctionsinvolvingvolatilestateorthelog(basicallyeverythingexceptTimerstuff)
• PartII– Correctlyhandlepersistentstate
19
![Page 20: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/20.jpg)
PartI:sendRequestVote
• MayhavealreadydoneforA3• SetRPCarguments
– lastLogIndex:lengthofthecandidate’slog(indexofcandidate’slastlogentry)
– lastLogTerm:ifwehavemorethanoneentry,termofthelastlogentry
20
![Page 21: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/21.jpg)
PartI:RequestVote
• Needtoaddcheckthatthecandidate’slogisatleastasup-to-dateasreceiver’slog
• Seesection5.4.1inoriginalpaperfordetails
21
![Page 22: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/22.jpg)
PartI:handleRequestVoteResponse• Ifwebecomeleader,initializenextIndex andmatchIndex– nextIndex:initializetothelengthoftheleader’slog(leaderlastlogindex+1)
– matchIndex:initializeto0(why?)
22
![Page 23: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/23.jpg)
PartI:sendAppendEntries
23
• Whichlogindexshouldwesendtofollowers?• IfourlastlogindexisgreaterthanorequaltothenextIndex forafollower,sendAppendEntries RPCwithlogentriesstartingatnextIndex
![Page 24: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/24.jpg)
PartI:AppendEntries
24
![Page 25: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/25.jpg)
PartI:handleAppendEntriesResponse
25
![Page 26: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/26.jpg)
PartII:persistingstate• OnlyneedtoreadfrompersistentstorageinMake• PersistwheneverwechangecurrentTerm,votedFor,orlog;
easy,right?• Thisbecomeshardifsimilarlogicissprinkledthroughoutyour
code.BesidesinMake,– logchangesinAppendEntries– votedFor changesduringelectionsandinAppendEntries
receive/handleresponse– currentTerm changeswheneverwebecomeFollower
• Notrequired,butforcompleteness– Shouldpersistbeforechanginginmemory;mostpeopledidnotdo
this(notethatyoudoneedtopersistbeforerespondingtoRPCs!)
26
![Page 27: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/27.jpg)
Somedetails• Locking/unlocking(whendowemodifystate?)
– Whenwehandletimeout– WhenwereceiveanRPC– WhenwehandleasingleRPCresponse– Start,Kill
• log,matchIndex,nextIndex– Youshouldreasonaboutthestateofthesearraysjustaswedidfor
Assignment3!– ManypeoplejuststartedimplementingbytranslatingFigure2into
code;withoutunderstanding,debuggingwillbemuchharder!
27
![Page 28: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/28.jpg)
Assignment5
28
![Page 29: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/29.jpg)
High-leveloverview
• ShouldonlydependonpublicRaftAPI• PartI:implementPut(key,value),Append(key,value),Get(key)– Musthavesequentialconsistency!
• Part2:handlingfailures– Dealwithduplicaterequests
29
![Page 30: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/30.jpg)
common.go• Shouldberelativelyquick!• Whatadditionalfield(s)doweneedtoputinPutAppendArgs?
• WhataboutGetArgs?
30
![Page 31: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/31.jpg)
client.go
• WejustneedtoproperlyconstructtheRPCstotheserver– Get– PutAppend
• TheseshouldfolloweasilyoncewehavetheArg structuresfromcommon.go
31
![Page 32: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/32.jpg)
server.go
• ThehardpartJ
32
![Page 33: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/33.jpg)
Debugging
33
![Page 34: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/34.jpg)
Generaltips
• Godebuggingisn’tgreat• Ifyouuseprintstatements,makesureyouuseunbufferedoutput(i.e.,usestderr)
• Usego’splayground:https://play.golang.org/• Createsubsetsoftheevaluationtests• Testincrementally:
– Thinkaboutinvariantsandcreateappropriatetests
34
![Page 35: Raft assignments - Princeton University Computer Science · •Assignment 3: an example in designing an implementation •Assignment 4: expanding on the example •Assignment 5: avoiding](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f1e6e641f8b7a58080779/html5/thumbnails/35.jpg)
Goslowtogofast
35