CopyrightJeffSutherland1993-2021 1
THESCRUMPAPERS:NUT,BOLTS,ANDORIGINSOFANAGILEFRAMEWORK
JEFFSUTHERLAND
CO-CREATOROFSCRUM
30MAR2021,LINCOLNMA
CopyrightJeffSutherland1993-2021 2
Dedication.....................................................................................................................................................................................5
Introduction.................................................................................................................................................................................8
Forward:IkujiroNonakaandTheScrumWay..........................................................................................................11
Chapter 1: Introduction to Scrum ....................................................................................................................... 13
ScrumPrimerVersion1.2...................................................................................................................................................14
RollingoutAgileatalargeEnterprise...........................................................................................................................33
CapturingExtremeBusinessValue:1000%AnnualReturnonInvestmentinScrumTrainers..........43
Chapter 2: The First Scrum ................................................................................................................................. 46
AgileDevelopment:LessonsLearnedfromtheFirstScrum................................................................................47
Chapter 3: What is Scrum? The First Papers on the Scrum Development Process ............................................... 54
ScrumDevelopmentProcess.............................................................................................................................................55
Chapter 4: Getting Started with Scrum .............................................................................................................. 79
ScrumonLargeProjects:Distributed,OutsourcedScrum...................................................................................81
AgileCanScale:InventingandReinventingScruminFiveCompanies...........................................................83
PatientKeeperScrum:TheFirstEnterpriseScrum..................................................................................................91
Conclusions................................................................................................................................................................................92
DistributedScrum:AgileProjectManagementwithOutsourcedDevelopmentTeams..........................93
Cahpter 5: Distributed, Outsourced Scrum ......................................................................................................... 96
Context.........................................................................................................................................................................................99
FullyDistributedScrum:ReplicatingLocalProductivityandQualitywithOffshoreTeams..............111
ScrumandCMMILevel5:TheMagicPotionforCodeWarriors.....................................................................124
MatureAgilewithatwistofCMMI...............................................................................................................................141
Chapter 6: Scrum Metrics ................................................................................................................................. 154
Reporting Scrum Project Progress to Executive Management through Metrics .................................................. 155
Introduction...........................................................................................................................................................................155
CopyrightJeffSutherland1993-2021 3
TransparencyintoProjects.............................................................................................................................................155
ExecutiveDashboard..........................................................................................................................................................156
Conclusions.............................................................................................................................................................................163
Chapter 7: Scrum Tuning ................................................................................................................................... 164
Type B Scrum Continuous Flow: Advancing the State of the Art ........................................................................ 166
Introduction...........................................................................................................................................................................166
BackgroundonScrum........................................................................................................................................................166
ImprovingScrumImplementations.............................................................................................................................168
EnhancingScrumImplementations.............................................................................................................................169
ManagementChallengesforaContinuousFlowScrum......................................................................................171
Summary..................................................................................................................................................................................172
Postscript.................................................................................................................................................................................172
Future of Scrum: Creating a Scrum Company with a Type C All-At-Once Scrum ................................................. 174
Abstract....................................................................................................................................................................................174
1.ScrumEvolution..............................................................................................................................................................174
2.ScrumEvolutioninPractice.......................................................................................................................................176
2.TheFirstScrums–TeamScrumandContinuousFlowScrum...................................................................177
3.ContinuousFlowScrum...............................................................................................................................................179
4.TypeCScrum....................................................................................................................................................................184
5.Conclusions........................................................................................................................................................................199
Chapter 8: Case Studies .................................................................................................................................... 201
Ssh! We are adding a process… (at Google) ....................................................................................................... 202
1. Introduction .................................................................................................................................................. 202
2. First agile attempts ....................................................................................................................................... 203
2.1.Theguineapigprojects.............................................................................................................................................204
CopyrightJeffSutherland1993-2021 4
2.2.Thefirstprocesssteps...............................................................................................................................................205
2.3.Issuestoovercome.....................................................................................................................................................206
2.4.Workingwiththeremoteteam.............................................................................................................................207
3.Addingagility–onepracticeatatime...................................................................................................................207
4.Releaseexperience.........................................................................................................................................................209
5.Feedbackandnextsteps..............................................................................................................................................210
6.Thesecondversion.........................................................................................................................................................211
7.Wherearewegoingfromhere..................................................................................................................................213
8.Summary.............................................................................................................................................................................214
Chapter 9:. References ...................................................................................................................................... 215
Appendix I: ....................................................................................................................................................... 221
Scrum: A Pattern Language for Hyperproductive Software Development .......................................................... 221
HowdoesScrumWork?....................................................................................................................................................222
ThePatterns...........................................................................................................................................................................224
Sprint.........................................................................................................................................................................................224
Backlog.....................................................................................................................................................................................228
ScrumMeetings....................................................................................................................................................................230
Conclusion...............................................................................................................................................................................235
Acknowledgements.............................................................................................................................................................236
Index ................................................................................................................................................................ 237
CopyrightJeffSutherland1993-2021 5
DEDICATIONThisbookisdedicatedtotheGodfathersofScrum,TakeuchiandNonaka[1],whogaveScrumitsnameandhelpedcreatedaglobaltransformationofsoftwaredevelopment.In2011Scrumisusedinover75%ofAgileimplementationsworldwide.ManyothershavecontributedtothecreationofScrum:
• JimCoplienand theATTBellLabsPasteurProject1for thepaperon themostproductivesoftwaredevelopment team ever documented – the Borland Quattro Pro Project [2]. The first Scrum teamimplementedtheScrumdailymeetingafterreadingthispaper.
• Nobel Laureates Muhammad Yunus and the Grameen Bank for originating microenterprisedevelopment and the Accion International President’s Advisory Board, responsible for much ofmicroenterprisedevelopmentinthewesternhemisphere.AsamemberoftheAccionadvisoryboard,JeffSutherlandnoticedthestrategyforbootstrappingthepooroutofpovertyisamodelforfreeinghundredsof thousandsof softwaredevelopers fromdeveloperabusecausedbypoormanagementpractices.
• AlanKayandhisteamatXeroxParc[3]forinventingSmalltalk,themouse,thegraphicaluserinterface,thepersonal computer, theEthernet, and the laserprinter. Listening tohis insights on innovationinspiredthefirstScrumteamtogofrom“good”to“great”[4].
!!Over 10 years of Pasteur Project research at ATT Bell Labs shows that more than a few roles reduces communication saturation and cripples performance. !The first published paper on the Pasteur approach to document projects and communication patterns coming out of ATT Bell Labs was: Cain, B.G. and J.O. Coplien. “"!#$%&'()*&+!,-./0/1)%!20$1&**!3$+&%/45!,46/0$4-&47.” Proceedings of the Second Intl Cof on the Software Process. Berlin 25-26 Feb 1993. !
This was followed by over 200 published case studies, the most important of which was read by the first Scrum team at Easel Corp. in 1993 (preprint) and started the Daily Scrum meeting. Coplien, James O. “($0%)4+!8$97:)0&!;0)97*-)4*</.=!"!>&:!?$$@!)7!20$1&**A!
BC)%/7D!)4+!20$+C17/6/7D.” Proceedings of the Fifth Borland International Conference, Orlando FL Jun 1994.!
It took over 10 years to publish a full summary of the Pasteur work which led to the software patterns movement in a book: Coplien, James O., Harrison, Neil B. “E05)4/F)7/$4)%!2)77&04*!$9!"5/%&!8$97:)0&!G&6&%$.-&47.” Pearson Prentice Hall 2005. The book contains the Borland paper as Chapter 8 and details the specifics of the Pasteur communication flow analysis.!
A few years after publication of the previous work, the authors asked Jeff Sutherland to work with them to establish a Scrum Patterns Group. So the original Pasteur work that led to the Scrum Daily Meeting has another 10 years later led to publication of: Jeff Sutherland, James O. Coplien, and The Scrum Patterns Group. "810C-!2)77&04*=!H<&!8./0/7!$9!7<&!I)-&J” Pragmatic Bookshelf 2019.!
All of this is just a small piece of the massive research project that created Scrum and the Scrum Guide and every serious Scrum professional should be familiar with the basics.!
CopyrightJeffSutherland1993-2021 6
• ProfessorRodneyBrooks for launchingthestartup iRobot inspace leasedfromJeffSutherland.Hetaughtusthesubsumptionarchitecture[5],howtocreatesimplerulestoproducehighlyintelligentperformancefromcomplexadaptivesystems.
• ChristopherLangtonofLosAlamosLabsandtheSanteFeInstituteforcoiningtheterm“artificiallife”andshowingthatincreasingdegreesoffreedomuptotheedgeofchaoticbehavioracceleratedtheirevolution[6].Scrumfeels“chaotic”byintent,soastoacceleratesoftwareevolution.
• The French object-database developers working near the MIT campus at Graphael (later ObjectDatabases,thenMatisseSoftware)fordemonstratingfirstinLispandtheninC++theAgilepatternsofpair programming, radical refactoring, continuous integration, common ownership of code, worldclass user interface design, and other tips and tricks which Kent Bent used to create eXtremeProgrammingadecadelater.ThesewereallincorporatedintothefirstScrum.
• TheCreativeInitiativeFoundationfortheirworkwithSiliconValleyvolunteerstohelpmaketheworlda better place, the underlying motivation driving the founders of Scrum. This connected the Co-CreatorsofScrumwiththeearlysystemsthinkingofMITProfessorPeterSengewholaterwrote“TheFifthDiscipline.”
• Capers Jones and his productivity experts at Software Productivity Research who analyzed andreanalyzedtheoutputofearlyScrumteams,aswellasmanyofthesoftwareproductsbuiltwithScrumduring1994-2000[7].TheseanalysesallowedustoprovideamoneybackguaranteethatuserswoulddoubleproductivityduringthefirstmonthusingtoolscreatedbythefirstScrum.
• The first Scrum team – John Scumniotales (Scrum Master), Don Roedner (Product Owner), JeffMcKenna(SeniorConsultant),JoeKinsella(object-relationalmapping),LaurelGinder(QA),andthreeDanishdevelopers-GrzegorzCiepiel,BentIllum,andJohnLindgreen.Theyenduredrepeatedfailure,depressing analysis of these failures in front of their technical peers from other companies, andtranscendenceoftheirmissteps.TheywerethefirstScrumteamtoachievethehyperproductivestateforwhichScrumwasdesignedandtheirproduct,ObjectStudio,wasreportedasindustryleaderbycomputer trade journals. Little did they know that Scrum would be their greatest contribution,althoughObjectStudiostilllivesonasasuccessfulproductalmost20yearslater.
• PatientKeeper,Inc.,thefirstcompanytofullyimplementan“AllatOnce”orTypeCScruminvolvingtheentirecompanyinScrumpractice.ThisinnovationinprocessdesignhasbeendocumentedbyMaryandTomPoppendieckintheirbookonLeanSoftwareDevelopment[8].“Ifindthatthevastmajorityoforganizationsarestill tryingtodotoomuchstuff,andthusfindthemselvesthrashing.TheonlyorganizationIknowofwhichhasreallysolvedthisisPatientKeeper[9].”PatientKeeperwasthefirstcompanytoachieveahyperproductiverevenuestatedrivenbyScrumin2007.Revenuequadrupledfrom13Mto50Minoneyear.
• JimJohnson,CEOoftheStandishGroup,continuouslyshareddataonover500,000projectsets,eachwith8-25projectsovertheyearssinceScrumbegan.ThesedatacontinuallyshowedthatwhileAgileprojects (77%Scrum)weremoresuccessful than traditionalprojects,58%ofAgileprojects fail todeliver.Thedatainthe2018ChaosReport–DecisionLatencyTheory:It’sAllAbouttheInterval[10]showedclearlythatScrumsuccesswasprimarilydrivenbyshorteningdecisiontime.
CopyrightJeffSutherland1993-2021 7
• Last,butnot least,manyScrumpractitionersexperience thequalitywithoutaname (QWAN) - aphraseusedbyChristopherAlexander inhisbook“TheTimelessWayofBuilding”[11].Alexanderdescribesacertainqualitythatweseek,butwhichcannotbenamed.ThismaybethemostimportantfeatureofScrumandcanonlybespokenofasasetofcorevalues-openness, focus,commitment,courage,andrespect.Itcouldbeviewedasthe“speedoftrust”oroneofthesourcesof“ba”oftenseenonScrumteams.BaistheJapanesetermforthecreativeflowofinnovationdescribedbyTakeuchiandNonaka[12].
Thankstothereviewersofthetextwhoincludeamongmanyothers:
• TomPoppendieck
• HenrickKniberg
• RowanBunning
• CliffordThompson
CopyrightJeffSutherland1993-2021 8
INTRODUCTIONScrum derives from complex adaptive systems theory and was influenced by best practices in Japaneseindustry,particularlyby leandevelopmentprinciples [13] implementedat companies likeToyota [14]andHonda [15], and knowledgemanagement strategies developed by Takeuchi andNonaka [12], now at theHitotsubashiBusinessSchoolinJapan,andPeterSenge[16]atMIT.ItwasenhancedbythepatternsmovementwhichevolvedoutofthePasteurProjectatATTBellLabsledbyJimCoplien.
Pasteur was a research program at Bell Labs based on sociometric analyses of over 100 organizations over a ten-year period. It went on in parallel with the development of Scrum, and in fact some of its findings provided the foundations for some elements of Scrum. One main thrust of the research was to create, catalog, and analyze sociograms built empirically from data extracted from production teams world-wide. These sociograms relate to Moreno's techniques for modeling social structures. Personal Communication (email): James Coplien. 22 Jun 2011
Scrumisnotadevelopmentmethodoraformalprocess,ratheritisacompressionalgorithmforworldwidebest practices observed in over 50 years of software development. The Scrum framework is simple toimplementandautomaticallyunpacksandencouragesasoftwaredevelopmentteamtodeploybestpracticesdocumented in Organizational Patterns of Agile Development. Author, Jim Coplien, comments, “Scrumencapsulates33ofthe45patternsinmybook.IttakestwominutesformetoexplainScrumandover60pagestoexplainthepatterns.ThiscompressionofbestpracticesisanamazingcharacteristicofScrum.”
AnotherunusualaspectofScrumisthatitworksinanydomain.JeffSutherlandisacoachtoOpenViewVenturePartnersandtheyruntheirinvestmentpracticeusingScrumwithdailystandupmeetings.JeffcoachesreligiousorganizationswhofindthatScrumradicallyimprovestheirprograms,finances,andnewmembership.Scrumhasbeenusedincompaniesacrossbusinessdomainsandevenforindividualfamiliesinplanningweddings,familychores,andchildren’sschedules.Itisasignificantinnovationinthewaytogetthingsdonefasterwithhigherqualitywhilemakingtheworkexperiencemorerewardingforallparticipants.
ScrumisusedasanAgilepracticethatdeliverssoftwaretoendusersfaster,better,andcooler[17,18].AstheChiefProductOwneratYahooobserved,coolnessisarequirementatGoogle,Yahoo,andmostsoftwaregamecompanies. Scrumsupportsa creativeapproach todevelopmentof complexand innovative systemsand itscalestolargenumbersofdevelopers.Itisusedonsomeoftheworld'slargestprojectsatcompanieslikeBritishTelecomorSiemensbecauseofhighproductivitywithlargedistributedandoutsourceddevelopmentteams.Itistheonlysoftwaredevelopmentprocessthathasdemonstratedlinearlyscalablewhenaddingresourcestolargeprojects [19,20]. Inaproperly implementedScrum,productivityperdeveloperstays thesamewhenaddingresources,aphenomenaneverseenbeforeinsoftwaredevelopment.
Themostprofitablesoftwareproductevercreated(GoogleAdwords[21])ispoweredbyScrumandthemostproductivelargeprojectwithoveramillionlinesofcode(SirsiDynix[19])usedadistributed,outsourcedScrumimplementation.CMMILevel5companiescutcostsinhalfwithScrumwhilesimultaneouslyimprovingquality,customersatisfaction,andthedeveloperexperience(SystematicSoftwareEngineering[20]).Atthesametime,Scrumremains theprocessof choice in small entrepreneurial companieswhere ithas its roots.OpenViewVenturePartners inBoston investsonly inAgileorganizationsandScrumis thecoreprocessused in theirportfoliocompanies.
The first software Scrumwas created at Easel Corporation [22] in 1993 based on extensive research onsuccessful projectsworldwide, a deep analysis of the computer science literature, close collaborationwith
CopyrightJeffSutherland1993-2021 9
leadingproductivityexperts,anddecadesofexperiencewithadvancedsoftwaretechnologies.JeffSutherlandwastheChiefEngineerfortheObjectStudioteamthatdefinedScrumroles,hiredthefirstProductOwnerandScrumMaster,developedthefirstProductBacklogandSprintBacklogandbuiltthefirstportfolioofproductscreatedwithScrum.
In1995,JeffintroducedtheScrumteamtoKenSchwaber,CEOofAdvancedDevelopmentMethods.KenagreedthatScrumwasabetterwaytobuildsoftwarethantraditionalmethodsusedatIBMandthebigeightconsultingfirmsandworkedwithJefftoformalizetheScrumdevelopmentprocessatOOPSLA’95[23].Inthesameyear,Sutherland provided support for development of eXtreme Programming [24] by giving Kent Beck allbackgroundinformationonthecreationofScrum[25]andtheresultsoftwoyearsofproductdevelopmentwiththeScrumprocessfrom1993-95.XPengineeringpracticesevolvedalongwithScrumandthetwoleadingAgile development processesworkwell together. Scrum and XP are themostwidely used Agile practicesworldwideandtheircreatorsaresignatoriesoftheAgileManifesto.
Agiledevelopmentisusedgloballyasthebestwaytodevelop,maintain,andsupportsoftwaresystems.SeveralpapersontheearlyimplementationofScrumareofgeneralinterest.Laterpapersprovidesomeofthenuts,bolts,andbestpracticesofScrumimplementations.ThedesignandimplementationofanAll-at-OnceScrum(TypeCScrum)atPatientKeepertoenableenterpriseagilityhasbeenemulatedbyinnovativecompaniesinmany countries. Case studies of CMMI Level 5 Scrum implementations and hyperproductive distributed,outsourcedteamsareofparticularinterest.Inthisbook,knowledgegainedfromthesestudiesareorganizedintoasinglevolumetobereadilyaccessible.
ScrumhasmadeitswaythroughthePatternLanguagesofProgrammingDesign(PLoP)process.BothScrumand eXtreme Programming were affected by the software patterns movement and Mike Beedle, a ScrumsignatoryoftheAgileManifesto,ledtheefforttoformallycodifyScrumasanorganizationalpattern.HisworkpublishedinVolume4ofPatternLanguagesofProgramDesign[26]inincludedhere.
Scrumisdesignedtoaddenergy,focus,clarity,andtransparencytoprojectplanningandimplementation.Itwillconsistently:
• Increasespeedofdevelopment
• Alignindividualandcorporateobjectives
• Createaculturedrivenbyperformance
• Supportshareholdervaluecreation
• Achievestableandconsistentcommunicationofperformanceatalllevels
• Enhanceindividualdevelopmentandqualityoflife
TheglobalexpansionofScruminboththelargestandsmallestsoftwarecompaniesandacrossallculturesisatestimonytothefactthatScrumdeliversonitspromise.WhileitisoftensaidthatScrumisnotasilverbullet,Scrumcanbelikeaheatseekingmissilewhenpointedintherightdirection.It’sinspectandadaptapproachtocontinuousqualityimprovementcandoseriousdamagetooutmodedbusinesspracticesandmanycompaniesarenowtransformingentireorganizationstotakeadvantageofScrumproductivity,todelightcustomers,andtomaketheworkenvironmentbetterandmorefunfordevelopmentteams.It’sfocusonbuildingcommunities
CopyrightJeffSutherland1993-2021 10
ofstakeholders,encouragingabetterlifefordevelopers,anddeliveringextremebusinessvaluetocustomers,releasescreativityandteamspiritinpractitionersandmakestheworldabetterplacetoliveandwork.
ForeverymemberofthefirstScrumteamandformanyteamsthatfollowed,individualliveswerechangedbyScrum.InNorwayin2008,themanagingdirectorofaScrumcompanycommentedthatannualrevenuewasquadrupledbyScrum.Shesaidshewasveryhappyaboutthesuccessofthecompanybutthebestthingwasthatthepeopleinthecompanyfeltsogoodaboutthewaytheyworkedtogether.Shefeltdeeplygratefulforthepositivefeelingofthepeople.SoScrumisasmuchabouttheheartasabouttheintellectandmoreaboutgettingpeopletohelponeanotherthanaboutprojectmanagement.Itcanfocusthespiritoftheteamandallowthemtotaketheirworktoahigherlevelwhereeveryonebenefitsintheprocessofdeliveringrealvaluetoeachother,managers,customers,andendusers.
SomepeopleonthefirstScrumteamwereconcernedtheywouldbesearchingfortherestoftheirlivestofindanotherteamthatwouldprovideasimilarexhilaratingfeelingofworkingtogethertoachievegreatsuccess.TheyfearedtheywouldneverbeabletofindanotheropportunityandthefirstScrumteamwouldbetheonlyworkexperienceintheirlivesthatdeeplysatisfiedthem.TheycriedinmyofficeaboutthiswhenthefirstScrumcompanywasacquiredbyalargercompanyandmovedtoanewlocation.Yetbecauseoftheirgenerosityofspirit,theirgiftofblood,sweat,andtears,andthecarefulshepherdingofScrumbyKenSchwaber,theycanrelax. Scrum teams are found everywhere, from Silicon Valley to Katmandu. All people now have theopportunitytoexperiencethebenefitsofScrum.
JeffSutherland-Lincon,MAUSA,2021
CopyrightJeffSutherland1993-2021 11
FORWARD:IKUJIRONONAKAANDTHESCRUMWAYbyJeffSutherland,Paris,2011
In 1993, the first Scrum team read “TheNew NewProductDevelopmentGame” in theHarvardBusinessReview.Theauthors,TakeuchiandNonaka,werevisitingprofessorsatHarvardwhentheypublishedthispaperin1986.Theydescribedthebestteamstheyhadseenaroundtheworldasself-organizingteamscharacterizedbyautonomy,transcendence,andcross-fertilization.Thatistheteammembersweretotallyresponsible,puttheteamfirst,andeveryonelearnedhowtodoeveryoneelse’sjob.Theteamleaderhadafacilitativestyle,helpedtheteamtranslateoutrageousseniormanagementgoalsintopracticalimplementation,andbecameacatalystforinnovationandorganizationaltransformation.
OurfirstScrumteamhadstudiedhundredsofpapersandtalkedwithmanyoftheleadingexpertsonsoftwaredevelopment and product management. The team had an outrageous goal ,to replace all the company’sproductswithabrandnewproductinlessthan6months.Noonecouldtellthemwhattodobecausenooneknewwhattheproductshouldbe.Itwouldbedifferentanditwouldbegroundbreakingandwewouldfigureitout.Nonaka’sworkwasaliferaftinastormyoceanandweallgrabbeditandheldon.
Seventeenyearslaterin2011,ImetNonakaforthefirsttimeinTokyo.Heisasmallman,ayoda-likefigurewhodoesnotuseacomputer.EarlyinhiscareerheworkedwiththemilitarytofigureoutwhytheJapaneselostWorldWarII.HeisnowoneofthemostinfluentialmanagementgurusontheplanetandsomesayhehasreplacedPeterDruckerastheleadingthinkerinmanagementinnovation.Hislatestbook,“ManagingFlow:AProcessTheoryoftheKnowledge-BasedFirm”isbasedonadeepunderstandingofphilosophyandhumanpsychology.Initheelucidatesthemagicthroughwhichgreatproductsarisefromthedynamicinteractionofateam.
Hisprimaryconcernin2011istohelpJapanthroughitsthirdgreattransition.ThefirstwaswhentheShogunturned Japan over to the Emperor because once Admiral Perry showed upwith guns, he could no longermaintainthelengthyefforttokeepJapanisolatedandimpervioustowesterninfluences.ThesecondwastheaftermathofHiroshimaandtherebuildingofJapan.NowJapanhasthehighestpercapitadebtintheworld.ThegrowthoftheireconomyisoneofthelowestinAsiaandtheChineseandtheSouthKoreansaredramaticeconomicthreats.Thepeopleallsaythereneedstobeandtherewillbeearth-shakingchangesinJapan,andsoon.
NonakaknewnothingaboutsoftwareScrumuntilaboutayearagowhenanemergingAgilecommunityinJapanattendedAgile2009.KawaguchiYasunobuandKenjiHiranabeorganizedthefirstAgileconferenceinJapanandaskedNonakatocomeandtalkaboutScrum.Fortheirsecondconference,theyinvitedmetomeetNonaka.whowassurprisedbyScrumandfounditdifferentfromhismeetingswiththeleadingmanagementandleangurus.HesaidhefelthumilityfromScrumratherthanthearrogancehehadexperiencedelsewhere.Hewassohappy that his ideas that developed in a university setting had achieved such widespread practicalimplementationinthebusinessworld.HefeelsdeeplythatJapanneedsradicalinnovationinthiscriticaltimeforhiscountryandthatScrumcanhelp.Atdinnerhesaidquietly,“HelpussaveJapan.”
IwasdeeplyaffectedbyNonakaandbythedozenorsoJapaneseScrumMasterswhoallwanttobecomeScrumTrainersandarereadytodoanythingtoachieveit,includinglearningEnglishsotheycanappearbeforethereviewcommitteeinAmerica.Duringourproductownercourse,theykeptaskingmetodrawakanjicharacter,repeatedlygivingmeparchment,acalligraphypen,andaninkbottle.TheysaiditwasimportantthatIpractice.Notknowingwhat thecharactermeant, Iwentalongwith them.Then lateatnightatdinner inabarafter
CopyrightJeffSutherland1993-2021 12
everyonehadhalfadozenbeers,theysaid,“Nowisthetime.YoumustdrawtheKanjicharacterforrealnow.”ThoughIdrewapoorimitationoftheexcellentexampletheygavemetofollow,theywerequitehappywithit.Theysaidatlast,“thismeans“TheScrumWay”andatthebottomisyoursignature.”
SuddenlyallmyyearsoftraininginAikidodojoscamebacktome.MasterUeshiba,thefounder,haddrawnthekanjifor“TheAikidoWay”andIhadstudiedundersomeofhisdirectdisciples.TheWayisawayofdoing,awayofbeing,itisawayoflife.Anditrequiresunendingpracticeinacommunityofpractitioners,withrelentlessfocus on continuous improvement. The “Way” kanji characters appear on the t-shirts, the kimonos, thepublications,anddoorwaystothehallsofthosewhopracticethe“Way.”IsawScrumasifforthefirsttimethroughtheeyesoftheJapanesewhendrawingtheiconof“TheScrumWay”andwasastounded!
CopyrightJeffSutherland1993-2021 13
CHAPTER1:INTRODUCTIONTOSCRUMTheScrumFoundationprovidedanintroductiontoScrumfirstdevelopedatYahooandlaterupdatedbyPeteDeemerandGabrielleBenefield fortheir independentconsultingpractices.TobiasMeyeraskedJeff tohelpeducateengineersatYahooonScruminNovember,2005.VicePresidentofDevelopment,PeteDeemersetupaYahooSeniorManagementmeetingthefollowingmonthforaScrumbriefingataneveningdinnerinPaloAlto.Bytheendofthedinner,YahoomanagementdecidedtorolloutScrumcompanywide.TheyfeltScrumfittheYahoostyleofdevelopmentusedintheirearlyyearsasastartupcompanywhilegivingthemastructurethatwouldsupportaglobalorganization.
PeterecruitedGabrielleBenefieldin2006toleadtheScrumrolloutatYahoointheU.S.andthenmovedtoIndiatobecomeYahooChiefProductOfficerandtraintheIndianteamsinScrum.Together,theyhavewrittentheirintroductiontoScrumandprovidedsomeinterestingsurveydata.In2007,Gabriellepublishedupdatedsurveydatafromalmost200ScrumteamsatYahoo.ThesedatashowoverwhelmingsupportforScrumasa“better,faster,cooler”methodforbuildingsoftware.ItalsoshowsanannualReturnonInvestment(ROI)of1000% for investment in Scrum trainers. Thiswas the estimatedROI on Scrum trainingproposed toPeteDeemerin2005byJeffSutherland.
Jeff,Pete,andGabrielle,alongwithJensOstergaardfromDenmarkjoinedforcestocreatetheScrumFoundation(formerly Scrum Training Institute) in 2008. This allows them to provide global training, consulting, andcoaching services to those companies implementingScrum.TheScrumFoundationexpanded to includeaspartnersmanyoftheleadingScrumcompanies,ScrumtrainersandScrumcoachesworldwide.
Inrecentyears,thefoundersfocusedontheirowncompanies,dissolvedtheScrumFoundationasacompany,andJensOstergaardnowownsthenameScrumFoundation.PeteDeemeralsoupdatedthisPrimer,addedCraigLarmanandBasVoddeasco-authors,andmaintainstheupdatedversionatGoodAgile.com.
CopyrightJeffSutherland1993-2021 14
SCRUMPRIMERVERSION1.2
PeteDeemerandGabrielleBenefield,ScrumFoundation,2010
Anotetoreaders:TherearemanyconcisedescriptionsofScrumavailableonline,andthisprimeraimstoprovidethenextlevelofdetailonthepractices.ItisnotintendedasthefinalstepinaScrumeducation;teamsthat are considering adopting Scrum are advised to equip themselves with Ken Schwaber’s Agile ProjectManagementwithScrumorAgileSoftwareDevelopmentwithScrum,andtakeadvantageofthemanyexcellentScrumtrainingandcoachingoptionsthatareavailable.OurthanksgotoKenSchwaber,Dr.JeffSutherland,andMikeCohnfortheirgenerousinput.
ãJeffSutherland1993-2021 15
TRADITIONALSOFTWAREDEVELOPMENT Thetraditionalwaytobuildsoftware,usedbycompaniesbigandsmall,wasasequentiallifecyclecommonlyknownas“thewaterfall.”Therearemanyvariants(suchastheV-Model),butittypicallybeginswithadetailedplanningphase,wheretheendproductiscarefullythoughtthrough,designed,anddocumentedingreatdetail.Thetasksnecessarytoexecutethedesignaredetermined,andtheworkisorganizedusingtoolssuchasGanttchartsandapplicationssuchasMicrosoftProject.Theteamarrivesatanestimateofhowlongthedevelopmentwilltakebyaddingupdetailedestimatesoftheindividualstepsinvolved.Oncestakeholdershavethoroughlyreviewed the plan and provided their approvals, the team starts towork. Teammembers complete theirspecializedportionofthework,andthenhanditoff toothers inproduction-linefashion.Oncetheworkiscomplete,itisdeliveredtoatestingorganization(somecallthisQualityAssurance),whichcompletestestingpriortotheproductreachingthecustomer.Throughouttheprocess,strictcontrolsareplacedondeviationsfromtheplantoensurethatwhatisproducedisactuallywhatwasdesigned.
Thisapproachhasstrengthsandweaknesses.Itsgreatstrengthisthatitissupremelylogical–thinkbeforeyoubuild,writeitalldown,followaplan,andkeepeverythingasorganizedaspossible.Ithasjustonegreatweakness:humansareinvolved.
Forexample,thisapproachrequiresthatthegoodideasallcomeatthebeginningofthereleasecycle,wheretheycanbeincorporatedintotheplan.Butasweallknow,goodideasappearthroughouttheprocess–inthebeginning,themiddle,andsometimeseventhedaybeforelaunch,andaprocessthatdoesnotpermitchangewillstiflethisinnovation.Withthewaterfall,agreatidealateinthereleasecycleisnotagift,it’sathreat.
The waterfall approach also places a great emphasis on writing things down as a primary method forcommunicatingcriticalinformation.TheveryreasonableassumptionisthatifIcanwritedownonpaperasmuchaspossibleofwhat’sinmyhead,itwillmorereliablymakeitintotheheadofeveryoneelseontheteam;plus,ifit’sonpaper,thereistangibleproofthatI’vedonemyjob.Thereality,though,isthatmostofthetimethese highly detailed 50-page requirements documents just do not get read.When they do get read, themisunderstandingsareoftencompounded.Awrittendocumentisanincompletepictureofmyideas;whenyoureadit,youcreateanotherabstraction,whichisnowtwostepsawayfromwhatIthinkImeanttosay atthattime.Itisnosurprisethatseriousmisunderstandingsoccur.
Somethingelsethathappenswhenyouhavehumansinvolvedisthehands-on“aha”moment–thefirsttimethatyouactuallyusetheworkingproduct.Youimmediatelythinkof20waysyoucouldhavemadeitbetter.Unfortunately,theseveryvaluableinsightsoftencomeattheendofthereleasecycle,whenchangesaremostdifficultanddisruptive–inotherwords,whendoingtherightthingismostexpensive,atleastwhenusingatraditionalmethod.
Humansarenotabletopredictthefuture.Forexample,yourcompetitionmakesanannouncementthatwasnotexpected.Unanticipatedtechnicalproblemscropupthatforceachangeindirection.Furthermore,peopleareparticularlybadatplanninguncertainthingsfarintothefuture–guessingtodayhowyouwillbespendingyourweekeightmonths fromnow is somethingofa fantasy. Ithasbeen thedownfallofmanya carefullyconstructedGanttchart.
In addition, a sequential life cycle tends to foster an adversarial relationship between the people that arehandingworkofffromonetothenext.“He’saskingmetobuildsomethingthat’snotinthespecification.”“She’schanginghermind.”“Ican’tbeheldresponsibleforsomethingIdon’tcontrol.”Andthisgetsustoanotherobservationaboutsequentialdevelopment–itisnotmuchfun.Thewaterfallmodelisacauseofgreatmiseryforthepeoplewhobuildproducts.Theresultingproductsfallwellshortofexpressingthecreativity,skill,and
ãJeffSutherland1993-2021 16
passionoftheircreators.Peoplearenotrobots,andaprocessthatrequiresthemtoactlikerobotsresultsinunhappiness.
Arigid,change-resistantprocessproducesmediocreproducts.Customersmaygetwhattheyfirstaskfor(atleasttwotranslationstepsremoved),butisitwhattheyreallywantoncetheyseetheproduct?Bygatheringalltherequirementsupfrontandhavingthemsetinstone,theproductiscondemnedtobeonlyasgoodastheinitialidea,insteadofbeingthebestoncepeoplehavelearnedordiscoverednewthings.
Manypractitionersofasequentiallifecycleexperiencetheseshortcomingsagainandagain.But,itseemssosupremelylogicalthatthecommonreactionistoturninward:“Ifonlywediditbetter,itwouldwork”–ifwejustplannedmore,documentedmore,resistedchangemore,everythingwouldworksmoothly.Unfortunately,manyteamsfindjusttheopposite:thehardertheytry,theworseitgets!Therearealsomanagementteamsthathaveinvestedtheirreputation–andmanyresources–inawaterfallmodel;changingtoafundamentallydifferentmodelisanapparentadmissionofhavingmadeamistake.AndScrumisfundamentallydifferent...
AGILEDEVELOPMENTANDSCRUMTheagilefamilyofdevelopmentmethodswerebornoutofabeliefthatanapproachmoregroundedinhumanreality–andtheproductdevelopmentrealityoflearning,innovation,andchange–wouldyieldbetterresults.Agileprinciplesemphasizebuildingworkingsoftwarethatpeoplecangethandsonquickly,versusspendingalotoftimewritingspecificationsupfront.Agiledevelopmentfocusesoncross-functionalteamsempoweredtomakedecisions,versusbighierarchiesandcompartmentalizationbyfunction.Anditfocusesonrapiditeration,withcontinuouscustomerinputalongtheway.OftenwhenpeoplelearnaboutagiledevelopmentorScrum,there’saglimmerofrecognition–itsoundsalotlikebackinthestart-updays,whenwe“justdidit.”
ByfarthemostpopularagilemethodisScrum.Itwasstronglyinfluencedbya1986HarvardBusinessReviewarticle on the practices associated with successful product development groups; in this paper the terms“Rugby” and “Scrum”were introduced,which latermorphed into “Scrum” inWicked Problems, RighteousSolutions (1991,DeGraceandStahl)relatingsuccessfuldevelopmentto thegameofRugby inwhichaself-organizingteammovestogetherdownthefieldofproductdevelopment.Itwasthenformalizedin1995byKenSchwaber and Dr. Jeff Sutherland. Scrum is now used by companies large and small, including Yahoo!,Microsoft,Google,LockheedMartin,Motorola,SAP,Cisco,GE,CapitalOneandtheUSFederalReserve.ManyteamsusingScrumreport significant improvements, and in somecases complete transformations, inbothproductivityandmorale.Forproductdevelopers–manyofwhomhavebeenburnedbythe“managementfadofthemonthclub”–thisissignificant.Scrumissimpleandpowerful.
SCRUMSUMMARYScrum is an iterative, incremental framework for projects and product or application development. ItstructuresdevelopmentincyclesofworkcalledSprints.Theseiterationsarenomorethanonemontheach,and takeplaceoneafter theotherwithoutpause.TheSprintsare timeboxed– theyendona specificdatewhethertheworkhasbeencompletedornot,andareneverextended.AtthebeginningofeachSprint,across-functionalteamselectsitems(customerrequirements)fromaprioritizedlist.TheteamcommitstocompletetheitemsbytheendoftheSprint. DuringtheSprint, thechosenitemsdonotchange.Everydaytheteamgathersbrieflytoinspectitsprogress,andadjustthenextstepsneededtocompletetheworkremaining.AttheendoftheSprint,theteamreviewstheSprintwithstakeholders,anddemonstrateswhatithasbuilt.PeopleobtainfeedbackthatcanbeincorporatedinthenextSprint.ScrumemphasizesworkingproductattheendoftheSprint that isreally“done”; in thecaseofsoftware, thismeanscodethat is integrated, fully testedandpotentiallyshippable.Keyroles,artifacts,andeventsaresummarizedinFigure1.
ãJeffSutherland1993-2021 17
AmajorthemeinScrumis“inspectandadapt.”Sincedevelopmentinevitablyinvolveslearning,innovation,andsurprises,Scrumemphasizestakingashortstepofdevelopment,inspectingboththeresultingproductandtheefficacyofcurrentpractices,andthenadaptingtheproductgoalsandprocesspractices.Repeatforever.
FIGURE1.SCRUM
SCRUMROLESInScrum,therearethreeroles:TheProductOwner,TheTeam,andTheScrumMaster. TogethertheseareknownasTheScrumTeam.TheProductOwnerisresponsibleformaximizingreturnoninvestment(ROI)byidentifyingproductfeatures,translatingtheseintoaprioritizedlist,decidingwhichshouldbeatthetopofthelistforthenextSprint,andcontinuallyre-prioritizingandrefiningthelist.TheProductOwnerhasprofitandlossresponsibilityfortheproduct,assumingitisacommercialproduct.Inthecaseofaninternalapplication,theProductOwnerisnotresponsibleforROIinthesenseofacommercialproduct(thatwillgeneraterevenue),buttheyarestillresponsibleformaximizingROIinthesenseofchoosing–eachSprint–thehighest-business-valuelowest-costitems.Inpractice,‘value’isafuzzytermandprioritizationmaybeinfluencedbythedesiretosatisfykeycustomers,alignmentwithstrategicobjectives,attackingrisks,improving,andotherfactors.Insomecases,theProductOwnerandthecustomerarethesameperson;thisiscommonforinternalapplications.Inothers,thecustomermightbemillionsofpeoplewithavarietyofneeds,inwhichcasetheProductOwnerroleissimilartotheProductManagerorProductMarketingManagerpositioninmanyproductorganizations.However,theProductOwnerissomewhatdifferentthanatraditionalProductManagerbecausetheyactivelyandfrequentlyinteractwiththeTeam,personallyofferingtheprioritiesandreviewingtheresultseachtwo-orfour-weekiteration,ratherthandelegatingdevelopmentdecisionstoaprojectmanager.Itisimportantto
ãJeffSutherland1993-2021 18
notethatinScrumthereisoneandonlyonepersonwhoservesas–andhasthefinalauthorityof–ProductOwner,andheorsheisresponsibleforthevalueofthework.
TheTeambuildstheproductthattheProductOwnerindicates:theapplicationorwebsite,forexample.TheTeaminScrumis“cross-functional”–itincludesalltheexpertisenecessarytodeliverthepotentiallyshippableproducteachSprint–and it is “self-organizing”(self-managing),withaveryhighdegreeofautonomyandaccountability.TheTeamdecideswhattocommitto,andhowbesttoaccomplishthatcommitment;inScrumlore,theTeamisknownas“Pigs”andeveryoneelseintheorganizationare“Chickens”(whichcomesfromajokeaboutapigandachickendecidingtoopenarestaurantcalled“HamandEggs,”andthepighavingsecondthoughtsbecause“hewouldbetrulycommitted,butthechickenwouldonlybeinvolved”).
TheTeaminScrumissevenplusorminustwopeople,andforasoftwareproducttheTeammightincludepeople with skills in analysis, development, testing, interface design, database design, architecture,documentation,andsoon.TheTeamdevelopstheproductandprovidesideastotheProductOwnerabouthowtomake theproduct great. In Scrum theTeamsaremostproductive andeffective if allmembers are100percentdedicatedtotheworkforoneproductduringtheSprint;avoidmultitaskingacrossmultipleproductsor projects. Stable teams are associated with higher productivity, so avoid changing Team members.Applicationgroupswithmanypeopleareorganized intomultipleScrumTeams,each focusedondifferentfeatures for the product, with close coordination of their efforts. Since one team often does all thework(planning,analysis,programming,andtesting)foracompletecustomer-centricfeature,Teamsarealsoknownasfeatureteams.
TheScrumMasterhelps theproductgroup learnandapplyScrumtoachievebusinessvalue. TheScrumMasterdoeswhateverisintheirpowertohelptheTeamandProductOwnerbesuccessful.TheScrumMasterisnotthemanageroftheTeamoraprojectmanager;instead,theScrumMasterservestheTeam,protectsthemfromoutside interference,andeducatesandguides theProductOwnerand theTeam in theskillfuluseofScrum.TheScrumMastermakes sureeveryone (including theProductOwner, and those inmanagement)understandsandfollowsthepracticesofScrum,andtheyhelpleadtheorganizationthroughtheoftendifficultchangerequiredtoachievesuccesswithagiledevelopment. SinceScrummakesvisiblemanyimpedimentsandthreatstotheTeam’sandProductOwner’seffectiveness,itisimportanttohaveanengagedScrumMasterworking energetically to help resolve those issues, or the Team or Product Ownerwill find it difficult tosucceed. Thereshouldbeadedicatedfull-timeScrumMaster,althoughasmallerTeammighthaveateammemberplaythisrole(carryingalighterloadofregularworkwhentheydoso).GreatScrumMasterscancomefromanybackgroundordiscipline:Engineering,Design,Testing,ProductManagement,ProjectManagement,orQualityManagement.
TheScrumMasterandtheProductOwnercannotbethesameindividual;attimes,theScrumMastermaybecalledupontopushbackontheProductOwner(forexample,iftheytrytointroducenewdeliverablesinthemiddleofaSprint).Andunlikeaprojectmanager,theScrumMasterdoesnottellpeoplewhattodoorassigntasks–theyfacilitatetheprocess,supportingtheTeamasitorganizesandmanagesitself.IftheScrumMasterwaspreviouslyinapositionmanagingtheTeam,theywillneedtosignificantlychangetheirmindsetandstyleofinteractionfortheTeamtobesuccessfulwithScrum.
Note there is no role of project manager in Scrum. This is because none is needed; the traditionalresponsibilities of a projectmanager have been divided up and reassigned among the three Scrum roles.Sometimesan(ex-)projectmanagercanstep into theroleofScrumMaster,but thishasamixedrecordofsuccess–thereisafundamentaldifferencebetweenthetworoles,bothinday-to-dayresponsibilitiesandinthemindsetrequiredtobesuccessful.
ãJeffSutherland1993-2021 19
Inadditiontothesethreeroles,thereareothercontributorstothesuccessoftheproduct,includingfunctionalmanagers(forexample,anengineeringmanager).WhiletheirrolechangesinScrum,theyremainvaluable.Forexample:
• theysupporttheTeambyrespectingtherulesandspiritofScrum• theyhelpremoveimpedimentsthattheTeamandProductOwneridentify• theymaketheirexpertiseandexperienceavailable
InScrum,theseindividualsreplacethetimetheypreviouslyspentplayingtheroleof“nanny”(assigningtasks,gettingstatusreports,andotherformsofmicromanagement)withtimeas“guru”and“servant”oftheTeam(mentoring,coaching,helpingremoveobstacles,helpingproblem-solve,providingcreativeinput,andguidingtheskillsdevelopmentofTeammembers).Inthisshift,managersmayneedtochangetheirmanagementstyle;forexample,usingSocraticquestioningtohelptheTeamdiscoverthesolutiontoaproblem,ratherthansimplydecidingasolutionandassigningittotheTeam.
STARTINGSCRUMThefirststepinScrumisfortheProductOwnertoarticulatetheproductvision.Eventually,thisevolvesintoarefinedandprioritizedlistoffeaturescalledtheProductBacklog.Thisbacklogexists(andevolves)overthelifetimeoftheproduct;itistheproductroadmap(Figure2).Atanypoint,theProductBacklogisthesingle,definitiveviewof“everythingthatcouldbedonebytheTeamever,inorderofpriority”.OnlyasingleProductBacklogexists;thismeanstheProductOwnerisrequiredtomakeprioritizationdecisionsacrosstheentirespectrum,representingtheinterestofstakeholdersandinfluencedbytheteam.
FIGURE2.THEPRODUCTBACKLOG
TheProductBacklogincludesavarietyofitems,primarilynewcustomerfeatures(“enablealluserstoplacebookinshoppingcart”),butalsoengineeringimprovementgoals(“reworkthetransactionprocessingmoduleto make it scalable”), exploratory or research work (“investigate solutions for speeding up credit cardvalidation”),and,possibly,knowndefects(“diagnoseandfixtheorderprocessingscripterrors”),ifthereareonlyafewproblems.(Asystemwithmanydefectsusuallyhasaseparatedefecttrackingsystem.)TheProductBacklogcanbearticulatedinanywaythatisclearandsustainable,thougheitherUseCasesor“userstories” areoftenusedtodescribetheProductBacklogitemsintermsoftheirvaluetotheenduseroftheproduct.
ãJeffSutherland1993-2021 20
ThesubsetoftheProductBacklogthatisintendedforthecurrentreleaseisknownastheReleaseBacklog,andingeneral,thisportionistheprimaryfocusoftheProductOwner.
TheProductBacklog is continuouslyupdatedby theProductOwner to reflect changes in theneedsof thecustomer,newideasorinsights,movesbythecompetition,technicalhurdlesthatappear,andsoforth.TheTeamprovidestheProductOwnerwithestimatesoftheeffortrequiredforeachitemontheProductBacklog.Inaddition,theProductOwnerisresponsibleforassigningabusinessvalueestimatetoeachindividualitem.ThisisusuallyanunfamiliarpracticeforaProductOwner.Assuch,itissomethingaScrumMastermayhelptheProductOwnerlearntodo.Withthesetwoestimates(effortandvalue)andperhapswithadditionalriskestimates, theProductOwnerprioritizes thebacklog (actually, usually just theReleaseBacklog subset) tomaximizeROI(choosingitemsofhighvaluewithloweffort)orsecondarily,toreducesomemajorrisk.Aswillbeseen,theseeffortandvalueestimatesmayberefreshedeachSprintaspeoplelearn;consequently,thisisacontinuousre-prioritizationactivityas theProductBacklogisever-evolving.
ScrumdoesnotdefinetechniquesforexpressingorprioritizingitemsintheProductBackloganditdoesnotdefineanestimationtechnique.Acommontechniqueistoestimateintermsofrelativesize(factoringineffort,complexity,anduncertainty)usingaunitof“storypoints”orsimply“points”.
Overtime,aTeamtrackshowmuchworkitcandoeachSprint;forexample,averaging26pointsperSprint.Withthisinformationtheycanprojectareleasedatetocompleteallfeatures,orhowmanyfeaturescanbecompletedbyafixeddate,iftheaveragecontinuesandnothingchanges.Thisaverageiscalledthe“velocity”oftheteam.VelocityisexpressedinthesameunitsastheProductBacklogitemsizeestimates.
TheitemsintheProductBacklogcanvarysignificantlyinsizeoreffort.LargeronesarebrokenintosmalleritemsduringtheProductBacklogRefinementworkshoportheSprintPlanningMeeting,andsmalleronesmaybeconsolidated.TheProductBacklogitemsfortheupcomingnextseveralSprintsshouldbesmallandfine-grainedenoughthattheyareunderstoodbytheTeam,enablingcommitmentsmadeintheSprintPlanningmeetingtobemeaningful;thisiscalledan“actionable”size.
OneofthemythsaboutScrumisthatitpreventsyoufromwritingdetailedspecifications;inreality,itisuptotheProductOwnerandTeamtodecidehowmuchdetailisrequired,andthiswillvaryfromonebacklogitemtothenext,dependingontheinsightoftheTeam,andotherfactors.Statewhatisimportantintheleastamountofspacenecessary–inotherwords,donotdescribeeverypossibledetailofanitem,justmakeclearwhatisnecessaryforittobeunderstood.Lowpriorityitems,farfrombeingimplementedandusually“coarsegrained”orlarge,havelessrequirementsdetails.Highpriorityandfine-graineditemsthatwillsoonbeimplementedtendtohavemoredetail.
SPRINTPLANNINGAtthebeginningofeachSprint,theSprintPlanningMeetingtakesplace.Itisdividedintotwodistinctsub-meetings,thefirstofwhichiscalledSprintPlanningPartOne.
InSprintPlanningPartOne,theProductOwnerandTeam(withfacilitationfromtheScrumMaster)reviewthehigh-priorityitemsintheProductBacklogthattheProductOwnerisinterestedinimplementingthisSprint.Theydiscussthegoalsandcontextforthesehigh-priorityitemsontheProductBacklog,providingtheTeamwithinsightintotheProductOwner’sthinking.TheProductOwnerandTeamalsoreviewthe“DefinitionofDone”(whichwasestablishedearlier) thatall itemsmustmeet, suchas, “Donemeanscodedtostandards,reviewed,implementedwithunittest-drivendevelopment(TDD),testedwith100percenttestautomation,integrated,anddocumented.”PartOnefocusesonunderstandingwhattheProductOwnerwants.According
ãJeffSutherland1993-2021 21
totherulesofScrum,attheendofPartOnethe(alwaysbusy)ProductOwnermayleavealthoughtheymustbeavailable(forexample,byphone)duringthenextmeeting.However,theyarewelcometoattendPartTwo...
SprintPlanningPartTwofocusesondetailedtaskplanning forhow to implementthe itemsthat theTeamdecidestotakeon.TheTeamselectstheitemsfromtheProductBacklogtheycommittocompletebytheendoftheSprint,startingatthetopoftheProductBacklog(inotherswords,startingwiththeitemsthatarethehighestpriorityfortheProductOwner)andworkingdownthelistinorder.ThisisakeypracticeinScrum:TheTeamdecideshowmuchworkitwillcommittocomplete,ratherthanhavingitassignedtothembytheProductOwner.ThismakesforamorereliablecommitmentbecausetheTeamismakingitbasedonitsownanalysisandplanning,ratherthanhavingitdecidedbysomeoneelse.WhiletheProductOwnerdoesnothavecontroloverhowmuchtheTeamcommitsto,heorsheknowsthattheitemstheTeamiscommittingtoaredrawn from the top of the Product Backlog – in otherwords, the items that he or she has rated asmostimportant.TheTeamhastheabilitytolobbyforitemsfromfurtherdownthelist;thisusuallyhappenswhentheTeamandProductOwnerrealizethatsomethingoflowerpriorityfitseasilyandappropriatelywiththehighpriorityitems.
TheSprintPlanningMeetingwilloftenlastanumberofhours,butnomorethaneighthoursforafour-weekSprint–theTeamismakingaseriouscommitmenttocompletethework,andthiscommitmentrequirescarefulthoughttobesuccessful.TheTeamwillprobablybegintheSprintPlanningPartTwobyestimatinghowmuchtimeeachmemberhasforSprint-relatedwork–inotherwords,theiraverageworkdayminusthetimetheyspendattendingmeetings,doingemail,takinglunchbreaks,andsoon.Formostpeoplethisworksoutto4-6hoursoftimeperdayavailableforSprint-relatedwork.Thisistheteam’scapacityfortheupcomingSprint.SeeFigure 3.
FIGURE3.ESTIMATINGAVAILABLEHOURS
Oncethecapacityisdetermined,theTeamfiguresouthowmanyProductBacklogitemstheycancompleteinthat time, and how they will go about completing them. This often starts with a design discussion at awhiteboard. Once theoveralldesign isunderstood, theTeamdecomposes theProductBacklog items intowork.TheTeamstartswiththefirstitemontheProductBacklog–inotherwords,theProductOwner’shighestpriorityitem–andworkingtogether,breaksitdownintoindividualtasks,whicharerecordedinadocumentcalledtheSprintBacklog(Figure4).
As mentioned, the Product Owner must be available during Part Two (such as via the phone) so thatclarificationispossible.TheTeamwillmovesequentiallydowntheProductBackloginthisway,untilit’susedupallitsestimatedcapacity.Attheendofthemeeting,theTeamwillhaveproducedalistofallthetaskswithestimates(typicallyinhoursorfractionsofaday).
Sprint Length 2 weeksWorkdays during Sprint 8 days
Tracy 8 4 32Sanjay 7 5 35Phillip 8 4 32Jing 6 5 30
* Net of vacation and other days out of office
Team Member
Available Days During
Sprint*
Available Hours per
Day
Total Available
Hours
ãJeffSutherland1993-2021 22
Scrumencouragesmulti-skilledworkers,ratherthanonly“workingtojobtitle”suchasa“tester”onlydoingtesting.Inotherwords,Teammembers“gotowheretheworkis”andhelpoutaspossible.Iftherearemanytestingtasks,thenallTeammembersmayhelp.Thisdoesnotimplythateveryoneisageneralist;nodoubtsomepeopleareespeciallyskilledintesting(andsoon)butTeammembersworktogetherandlearnnewskillsfromeachother.Consequently,duringtaskgenerationandestimationinSprintPlanning,itisnotnecessary–nor appropriate – for people to volunteer for all the tasks “they can do best.” Rather, it is better to onlyvolunteerforonetaskatatime,whenitistimetopickupanewtask,andtoconsiderchoosingtasksthatwillonpurposeinvolvelearning(perhapsbypairworkwithaspecialist).
Allthatsaid,thereareraretimeswhenJohnmaydoaparticulartaskbecauseitwouldtakefartoolongorbeimpossibleforotherstolearn–perhapsJohnistheonlypersonwithanyartisticskilltodrawpictures.OtherTeammemberscouldnotdrawa“stickman”iftheirlifedependedonit.Inthisrarecase–andifitisnotrareandnotgettingrarerastheTeamlearns,thereissomethingwrong–itmaybenecessarytoaskifthetotalplanneddrawingtasksthatmustbedonebyJohnarefeasiblewithintheshortSprint.
ManyTeamsalsomakeuseofavisualtask-trackingtool,intheformofawall-sizedtaskboardwheretasks(writtenonPost-ItNotes)migrateduringtheSprintacrosscolumnslabeled“NotYetStarted,”“InProgress,”and“Completed.”SeeFigure5.
FIGURE4.SPRINTBACKLOG
FIGURE5.VISUALMANAGEMENT-SPRINTBACKLOGTASKSONTHEWALL
New Estimates of EffortRemaining as of Day...
Product Backlog Item Sprint Task Volunteer 1 2 3 4 5 6
modify database 5
create webpage (UI) 8
create webpage (Javascript logic) 13
write automated acceptance tests 13
update buyer help webpage 3
. . .
merge DCP code and complete layer-level tests 5
complete machine order for pRank 8
change DCP and reader to use pRank http API 13
Initial Estimate of
Effort
As a buyer, I want to place a book in a shopping cart
Improve transaction processing performance
ãJeffSutherland1993-2021 23
OneofthepillarsofScrumisthatoncetheTeammakesitscommitment,anyadditionsorchangesmustbedeferreduntilthenextSprint.ThismeansthatifhalfwaythroughtheSprinttheProductOwnerdecidesthereisanewitemheorshewouldliketheTeamtoworkon,hecannotmakethechangeuntilthestartofthenextSprint.Ifanexternalcircumstanceappearsthatsignificantlychangespriorities,andmeanstheTeamwouldbewastingitstimeifitcontinuedworking,theProductOwnerortheTeamcanterminatetheSprint.TheTeamstops,andanewSprintPlanningmeetinginitiatesanewSprint.Thedisruptionofdoingthisisusuallygreat;thisservesasadisincentivefortheProductOwnerorTeamtoresorttothisdramaticdecision.
Thereisapowerful,positiveinfluencethatcomesfromtheTeambeingprotectedfromchanginggoalsduringtheSprint.First,theTeamgetstoworkknowingwithabsolutecertaintythatitscommitmentswillnotchange,thatreinforcestheTeam’sfocusonensuringcompletion.Second,itdisciplinestheProductOwnerintoreallythinkingthroughtheitemsheorsheprioritizesontheProductBacklogandofferstotheTeamfortheSprint.
By following theseScrumrules theProductOwnergains twothings.First,heorshehas theconfidenceofknowingtheTeamhasmadeacommitmenttocompletearealisticandclearsetofworkithaschosen. Overtime a Team can become quite skilled at choosing and delivering on a realistic commitment. Second, theProductOwnergetstomakewhateverchangesheorshelikestotheProductBacklogbeforethestartofthenext Sprint. At that point, additions, deletions, modifications, and re-prioritizations are all possible andacceptable.WhiletheProductOwnerisnotabletomakechangestotheselecteditemsunderdevelopmentduringthecurrentSprint,heorsheisonlyoneSprint’sdurationorlessawayfrommakinganychangestheywish.Goneisthestigmaaroundchange–changeofdirection,changeofrequirements,orjustplainchangingyourmind–anditmaybeforthisreasonthatProductOwnersareusuallyasenthusiasticaboutScrumasanyone.
DAILYSCRUMOncetheSprinthasstarted,theTeamengagesinanotherofthekeyScrumpractices:TheDailyScrum.Thisisashort(15minutesorless)meetingthathappenseveryworkdayatanappointedtime.EveryoneontheTeamattends.Tokeepitbrief, it isrecommendedthateveryoneremainstanding.It istheTeam’sopportunitytosynchronizetheirworkandreporttoeachotheronobstacles.IntheDailyScrum,onebyone,eachmemberoftheTeamreportsthree(andonlythree)thingstotheothermembersoftheTeam:(1)Whattheywereabletogetdonesincethelastmeeting;(2)whattheyareplanningtofinishbythenextmeeting;and(3)anyblocksorimpedimentsthatareintheirway.
NotethattheDailyScrumisnotastatusmeetingtoreporttoamanager;itisatimeforaself-organizingTeamtosharewitheachotherwhatisgoingon,tohelpthemcoordinate.Someonemakesnoteoftheblocks,andtheScrumMaster is responsible tohelpTeammembersresolve them.There isnodiscussionduring theDailyScrum,onlyreportinganswerstothethreequestions;ifdiscussionisrequiredittakesplaceimmediatelyaftertheDailyScruminafollow-upmeeting,althoughinScrumnooneisrequiredtoattendthis.Thisfollow-upmeetingisacommoneventwheretheTeamadaptstotheinformationtheyheardintheDailyScrum:inotherwords, another inspect and adapt cycle. It is generally recommended not to havemanagers or others inpositionsofperceivedauthorityattendtheDailyScrum.ThisrisksmakingtheTeamfeel“monitored”–underpressure to report major progress every day (an unrealistic expectation), and inhibited about reportingproblems–andittendstounderminetheTeam’sself-management,andinvitemicromanagement.ItwouldbemoreusefulforastakeholdertoinsteadreachouttotheTeamfollowingthemeeting,andoffertohelpwithanyblocksthatareslowingtheTeam’sprogress.
ãJeffSutherland1993-2021 24
UPDATINGSPRINTBACKLOG&SPRINTBURNDOWNCHARTTheTeaminScrumisself-managing,andinordertodothissuccessfully,itmustknowhowitisdoing.Everyday,theTeammembersupdatetheirestimateoftheamountoftimeremainingtocompletetheircurrenttaskintheSprintBacklog(Figure6).Followingthisupdate,someoneaddsupthehoursremainingfortheTeamasawhole,andplotsitontheSprintBurndownChart(Figure7).Thisgraphshows,eachday,anewestimateofhowmuchwork(measuredinpersonhours)remainsuntiltheTeam’stasksarefinished.Ideally,thisisadownwardslopinggraphthatisonatrajectorytoreach“zeroeffortremaining”bythelastdayoftheSprint.Henceitiscalledaburndownchart.Andwhilesometimesitlooksgood,oftenitdoesnot;thisistherealityofproductdevelopment.TheimportantthingisthatitshowstheTeamtheirprogresstowardstheirgoal,notintermsofhowmuchtimewasspentinthepast(anirrelevantfactintermsofprogress),butintermsofhowmuchworkremainsinthefuture–whatseparatestheTeamfromtheirgoal.IftheburndownlineisnottrackingdownwardstowardscompletionneartheendoftheSprint,thentheTeamneedstoadjust,suchastoreducethescopeoftheworkortofindawaytoworkmoreefficientlywhilestillmaintainingasustainablepace.
WhiletheSprintBurndownchartcanbecreatedanddisplayedusingaspreadsheet,manyTeamsfinditismoreeffectivetoshowitonpaperonawall intheirworkspace,withupdatesinpen;this“low-tech/high-touch”solutionisfast,simple,andoftenmorevisiblethanacomputerchart.
FIGURE6.DAILYUPDATESOFWORKREMAININGONTHESPRINTBACKLOG
ãJeffSutherland1993-2021 25
FIGURE7.SPRINTBURNDOWNCHART
PRODUCTBACKLOGREFINEMENTOneofthelesserknown,butvaluable,guidelinesinScrumisthatfiveortenpercentofeachSprintmustbededicatedbytheTeamtorefining(or“grooming”)theProductBacklog.Thisincludesdetailedrequirementsanalysis,splittinglargeitemsintosmallerones,estimationofnewitems,andre-estimationofexistingitems.Scrumissilentonhowthisworkisdone,butafrequentlyusedtechniqueisafocusedworkshopneartheendoftheSprint,sothattheTeamandProductOwnercandedicatethemselvestothisworkwithoutinterruption.
Foratwo-weekSprint,fivepercentofthedurationimpliesthateachSprintthereisahalf-dayProductBacklogRefinementworkshop.ThisrefinementactivityisnotforitemsselectedforthecurrentSprint;itisforitemsforthefuture,mostlikelyinthenextoneortwoSprints.Withthispractice,SprintPlanningbecomesrelativelysimplebecausetheProductOwnerandScrumTeamstarttheplanningwithaclear,well-analyzedandcarefullyestimatedsetofitems.Asignthatthisrefinementworkshopisnotbeingdone(ornotbeingdonewell)isthatSprintPlanninginvolvessignificantquestions,discovery,orconfusionandfeels incomplete;planningworkthenoftenspillsoverintotheSprintitself,whichistypicallynotdesirable.
ENDINGTHESPRINTOneofthecoretenetsofScrumisthatthedurationoftheSprintisneverextended–itendsontheassigneddateregardlessofwhethertheTeamhascompletedtheworkitcommittedto.ATeamtypicallyover-commitsinitsfirstfewSprintsandfailstoaccomplishitscommitments.Sometimesitthenovercompensatesandunder-commits,andfinishesearly(inwhichcaseitcanasktheProductOwnerformoreProductBacklogitemstoworkon).ButbythethirdorfourthSprintaTeamhastypicallyfiguredoutwhatitiscapableofdelivering(mostofthetime),andtheywillmeettheirSprintgoalsmorereliablyafterthat.TeamsareencouragedtopickonedurationfortheirSprints(say,twoweeks)andnotchangeit.ThishelpstheTeamlearnhowmuchitcanaccomplish,whichhelpsinbothestimationandlonger-termreleaseplanning.ItalsohelpstheTeamachievearhythmfortheirwork;thisisoftenreferredtoasthe“heartbeat”oftheTeaminScrum.
SPRINTREVIEWAftertheSprintends,thereistheSprintReview,wheretheTeamandtheProductOwnerreviewtheSprint.This isoftenmislabeledthe“demo”butthatdoesnotcapturethereal intentofthismeeting.AkeyideainScrumisinspectandadapt.Toseeandlearnwhatisgoingonandthenevolvebasedonfeedback,inrepeatingcycles.TheSprintReviewisaninspectandadaptactivityfortheproduct.ItisatimefortheProductOwnertolearnwhatisgoingonwiththeproductandwiththeTeam(thatis,areviewoftheSprint);andfortheTeamtolearnwhatisgoingonwiththeProductOwnerandthemarket.Consequently,themostimportantelementoftheReviewisanin-depthconversationbetweentheTeamandProductOwnertolearnthesituation,togetadvice,andsoforth.ThereviewincludesademoofwhattheTeambuiltduringtheSprint,butifthefocusofthereviewisademoratherthanconversation,thereisanimbalance.
Auseful–butoftenoverlooked–ScrumguidelineisthatittheScrumMaster’sresponsibilitytoensurethateveryone knows the “Definition of Done” defined for this product or release. He prevents the team fromdemonstratingordiscussingProductBacklogItemsthatarenot‘done’accordingtothe“DefinitionofDone.”Itemsthatarenot‘done’gobacktotheProductBacklogandwillbere-prioritizedbytheProductOwner.Inway, there is transparencyregardingthequalityof thework;Teamscannot fakethequalitybypresentingsoftwarethatappearstoworkwell,butmaybeimplementedwithamessypileofpoorqualityanduntestedcode.
ãJeffSutherland1993-2021 26
Present at this meeting are the Product Owner, Team members, and Scrum Master, plus customers,stakeholders,experts,executives,andanyoneelseinterested.ThedemoportionoftheSprintReviewisnota“presentation”theTeamgives–thereisnoslideware.AguidelineinScrumisthatnomorethan30minutesshouldbespentpreparingforthereview,otherwiseitsuggestssomethingiswrongwiththeworkoftheTeam.Itissimplyademoofwhathasbeenbuilt.Anyonepresentisfreetoaskquestionsandgiveinput.
SPRINTRETROSPECTIVETheSprintReviewinvolvesinspectandadaptregardingtheproduct.TheSprintRetrospective,whichfollowstheReview,involvesinspectandadaptregardingtheprocess. ThisisapracticethatsomeTeamsskip,andthat’sunfortunate,becauseit’sthemainmechanismfortakingthevisibilitythatScrumprovidesintoareasofpotentialimprovement,andturningitintoresults.It’sanopportunityfortheTeamtodiscusswhat’sworkingandwhat’snotworking,andagreeonchangestotry.TheTeamandScrumMasterwillattend,andtheProductOwneriswelcomebutnotrequiredtoattend.SometimestheScrumMastercanactasaneffectivefacilitatorfortheretrospective,butitmaybebettertofindaneutraloutsidertofacilitatethemeeting;agoodapproachisforScrumMasterstofacilitateeachothers’retrospectives,whichenablescross-pollinationamongTeams.
TherearemanytechniquesforconductingaSprintRetrospective,andthebookAgileRetrospectives(Derby,Larsen2006)providesausefulcatalogueoftechniques.Asimplewaytostructurethediscussionistodrawtwocolumnsonawhiteboard,labeled“What’sWorkingWell”and“WhatCouldWorkBetter”–andthengoaroundtheroom,witheachpersonaddingoneormoreitemstoeitherlist.Asitemsarerepeated,checkmarksareaddednexttothem,sothecommonitemsbecomeclear.ThentheTeamlooksforunderlyingcauses,andagreesonasmallnumberofchangestotryintheupcomingSprint,alongwithacommitmenttoreviewtheresultsatthenextSprintRetrospective.
AusefulpracticeattheendoftheRetrospectiveisfortheTeamtolabeleachoftheitemsineachcolumnwitheithera“C”ifitiscausedbyScrum(inotherwords,withoutScrumitwouldnotbehappening),oran“E”ifitisexposedbyScrum(inotherwords,itwouldbehappeningwithorwithoutScrum,butScrummakesitknowntotheTeam),ora“U”ifit’sunrelatedtoScrum(liketheweather).TheTeammayfindalotofC’sonthe“What’sWorkingWell”sideoftheboard,andalotofE’sonthe“WhatCouldWorkBetter”;thisisgoodnews,evenifthe“WhatCouldWorkBetter”listisalongone,becausethefirststeptosolvingunderlyingissuesismakingthemvisible,andScrumisapowerfulcatalystforthat.
UPDATINGRELEASEBACKLOG&BURNDOWNCHARTAtthispoint,someitemshavebeenfinished,somehavebeenadded,somehavenewestimates,andsomehavebeendroppedfromthereleasegoal.TheProductOwnerisresponsibleforensuringthatthesechangesarereflecting in the Release Backlog (andmore broadly, the Product Backlog). In addition, Scrum includes aRelease Burndown chart that shows progress towards the release date. It is analogous to the SprintBurndownchart,butisatthehigherlevelofitems(requirements)ratherthanfine-grainedtasks.SinceanewProductOwnerisunlikelytoknowwhyorhowtocreatethischart,thisisanotheropportunityforaScrumMastertohelptheProductOwner.SeeFigure8andFigure9foranexampleoftheReleaseBacklogandReleaseBurndownchart.
ãJeffSutherland1993-2021 27
FIGURE8.RELEASEBACKLOG(ASUBSETOFTHEPRODUCTBACKLOG)
FIGURE9.RELEASEBURNDOWNCHART
STARTINGTHENEXTSPRINTFollowingtheSprintReview,theProductOwnermayupdatetheProductBacklogwithanynewinsight.Atthispoint,theProductOwnerandTeamarereadytobeginanotherSprintcycle.ThereisnodowntimebetweenSprints –Teamsnormally go froma SprintRetrospective one afternoon into thenext Sprint Planning thefollowingmorning(oraftertheweekend).
One of the principles of agile development is “sustainable pace”, and only byworking regular hours at areasonablelevelcanTeamscontinuethiscycleindefinitely.
RELEASESPRINT
New Estimates of EffortRemaining at end of Sprint...
Item Priority 1 2 3 4 5 6
... 1 7 5 0 0 0
As a buyer, I want to remove a book in a shopping cart ... 2 6 2 0 0 0
... 3 6 13 13 0 0
... 4 6 20 20 20 0
Upgrade all servers to Apache 2.2.3 ... 5 5 13 13 13 13
... 6 2 3 3 3 3
As a shopper, I want to create and save a wish list ... 7 7 40 40 40 40
... 8 4 20 20 20 20
. . . . . . . . .
Total 537 580 570 500
Details (wiki URL)
Estimate of Value
Initial Estimate of Effort
As a buyer, I want to place a book in a shopping cart (see UI sketches on wiki page)
Improve transaction processing performance (see target performance metrics on wiki)
Investigate solutions for speeding up credit card validation (see target performance metrics on wiki)
Diagnose and fix the order processing script errors (bugzilla ID 14823)
As a shopper, I want to to add or delete items on my wish list
ãJeffSutherland1993-2021 28
TheperfectionvisionofScrumisthattheproduct ispotentiallyshippableat theendofeachSprint,whichimpliesthereisnowrapupworkrequired,suchastestingordocumentation.Theimplicationisthateverythingiscompletely finishedeverySprint;thatyoucouldactuallyshipitordeployit immediatelyaftertheSprintReview. This means that each increment is a complete slice of the final product and gives completetransparencytotheProductOwnerandstakeholders.TheyknowexactlywheretheyareattheendofeverySprint.
However,manyorganizationshaveweakdevelopmentpractices,toolsandinfrastructureandcannotachievethisperfectionvision,orthereareotherextenuatingcircumstances(suchas,“themachinebroke”).Inthiscase,therewillbesomeremainingwork,suchasfinalproductionenvironmentintegrationtesting,andsotherewillbetheneedfora“ReleaseSprint”tohandlethisremainingwork.
NotethattheneedforaReleaseSprintisasignofsomeweakness;theidealisthatitisnotrequired.Whennecessary,SprintscontinueuntiltheProductOwnerdecidestheproductisalmostreadyforrelease,atwhichpointtherewillbeaReleaseSprinttoprepareforlaunch.IftheTeamhasfollowedgooddevelopmentpractices,withcontinuousrefactoringandintegration,andeffectivetestingduringeachSprint,thereshouldbelittlepre-releasestabilizationorotherwrap-upworkrequired.
RELEASEPLANNING&INITIALPRODUCTBACKLOGREFINEMENTAquestionthatissometimesaskedishow,inaniterativemodel,canlong-termreleaseplanningbedone.Therearetwocasestoconsider:(1)anewproductinitsfirstrelease,and(2)anexistingproductinalaterrelease.
Inthecaseofanewproduct,oranexistingproductjustadoptingScrum,thereistheneedtodoinitialProductBacklogrefinementbeforethefirstSprint,wheretheProductOwnerandTeamshapeaproperScrumProductBacklog.Thiscouldtakeafewdaysoraweek,andinvolvesavisionworkshop,somedetailedrequirementsanalysis,andestimationofalltheitemsidentifiedforthefirstrelease.
SurprisinglyinScrum,inthecaseofanestablishedproductwithanestablishedProductBacklog,thereshouldnotbetheneedforanyspecialorextensivereleaseplanningforthenextrelease.Why?BecausetheProductOwnerandTeamshouldbedoingProductBacklogrefinementeverySprint(fiveortenpercentofeachSprint),continuouslypreparingforthefuture.Thiscontinuousproductdevelopmentmodeobviatestheneedforthedramatic punctuated prepare-execute-conclude stages one sees in traditional sequential life cycledevelopment.
DuringaninitialProductBacklogrefinementworkshopandduringthecontinuousbacklogrefinementeachSprint,theTeamandProductOwnerwilldoreleaseplanning,refiningtheestimates,priorities,andcontentastheylearn.
Somereleasesaredate-driven; forexample:“Wewillreleaseversion2.0ofourprojectatatrade-showonNovember10.”Inthissituation,theTeamwillcompleteasmanySprints(andbuildasmanyfeatures)asispossible inthetimeavailable.Otherproductsrequirecertainfeaturestobebuiltbeforetheycanbecalledcompleteandtheproductwillnotlaunchuntiltheserequirementsaresatisfied,howeverlongthattakes.SinceScrumemphasizesproducingpotentiallyshippablecodeeachSprint,theProductOwnermaychoosetostartdoinginterimreleases,toallowthecustomertoreapthebenefitsofcompletedworksooner.
Sincetheycannotpossiblyknoweverythingupfront,thefocusisoncreatingandrefiningaplantogivethereleasebroaddirection,andclarifyhowtradeoffdecisionswillbemade(scopeversusschedule,forexample).Thinkofthisastheroadmapguidingyoutowardsyourfinaldestination;whichexactroadsyoutakeandthedecisionsyoumakeduringthejourneymaybedeterminedenroute.
ãJeffSutherland1993-2021 29
MostProductOwnerschooseonereleaseapproach.Forexample,theywilldecideareleasedate,andwillworkwiththeTeamtoestimatetheReleaseBacklogitemsthatcanbecompletedbythatdate.Insituationswherea“fixedprice/fixeddate/fixeddeliverable”commitmentisrequired–forexample,contractdevelopment–oneormoreofthoseparametersmusthaveabuilt-inbuffertoallowforuncertaintyandchange;inthisrespect,Scrumisnodifferentfromotherapproaches.
APPLICATIONORPRODUCTFOCUSForapplicationsorproducts–eitherforthemarketorforinternalusewithinanorganization–Scrummovesgroups away from the older project-centric model toward a continuous application/product developmentmodel. There is no longer a project with a beginning, middle, and end. And hence no traditional projectmanager.Rather,thereissimplyastableProductOwnerandalong-livedself-managingTeamthatcollaborateinan“endless”seriesoffixed-lengthSprints,untiltheproductorapplicationisretired.Allnecessary“project”managementworkishandledbytheTeamandtheProductOwner–whoisaninternalbusinesscustomerorfromProductManagement.ItisnotmanagedbyanITmanagerorsomeonefromaProjectManagementOffice.
Scrumcanalsobeusedfortrueprojectsthatareone-timeinitiatives(ratherthanworktocreateorevolvelong-livedapplications);still,inthiscasetheTeamandProductOwnerdotheprojectmanagement.
Whatifthereisinsufficientnewworkfromoneormoreexistingapplicationstowarrantadedicatedlong-livedTeamforeachapplication?Inthiscase,astablelong-livedTeammaytakeonitemsfromoneapplicationinoneSprint,andthenitemsfromanotherinthenextSprint;inthissituationtheSprintsareoftenquiteshort,suchasoneweek.
Occasionally,thereisinsufficientnewworkevenforthepriorsolution,andtheTeammaytakeonitemsfromseveralapplicationsduringthesameSprint;however,bewarethissolutionasitmaydevolveintounproductivemultitaskingacrossmultipleapplications.AbasicproductivitythemeinScrumisfortheTeamtobefocusedononeproductorapplicationforoneSprint.
COMMONCHALLENGESScrumisnotonlyaconcretesetofpractices–rather,andmoreimportantly,itisaframeworkthatprovidestransparency, and a mechanism that allows “inspect and adapt”. Scrum works by making visible thedysfunctionandimpedimentsthatareimpactingtheProductOwnerandtheTeam’seffectiveness,sothattheycanbeaddressed.Forexample,theProductOwnermaynotreallyknowthemarket,thefeatures,orhowtoestimatetheirrelativebusinessvalue.OrtheTeammaybeunskillfulineffortestimationordevelopmentwork.
The Scrum framework will quickly reveal these weaknesses. Scrum does not solve the problems ofdevelopment;itmakesthempainfullyvisible,andprovidesaframeworkforpeopletoexplorewaystoresolveproblemsinshortcyclesandwithsmallimprovementexperiments.
SupposetheTeamfails todeliverwhat theycommittedto in the firstSprintduetopoortaskanalysisandestimationskill.TotheTeam,thisfeelslikefailure.Butinreality,thisexperienceisthenecessaryfirststeptowardbecomingmorerealisticandthoughtfulaboutitscommitments.Thispattern–ofScrumhelpingmakevisibledysfunction,enablingtheTeamtodosomethingaboutit–isthebasicmechanismthatproducesthemostsignificantbenefitsthatTeamsusingScrumexperience.
ãJeffSutherland1993-2021 30
Onecommonmistakemade,whenpresentedwithaScrumpracticethatischallenging,istochangeScrum.Forexample,Teams thathave troubledeliveringon theirSprintcommitmentmightdecide tomake theSprintdurationextendable,soitneverrunsoutoftime–andintheprocess,ensureitneverhastolearnhowtodoabetterjobofestimatingandmanagingitstime.Inthisway,withoutcoachingandthesupportofanexperiencedScrumMaster,organizationscanmutateScrumintojustamirrorimageofitsownweaknessesanddysfunction,and undermine the real benefit that Scrum offers: Making visible the good and the bad, and giving theorganizationthechoiceofelevatingitselftoahigherlevel.
AnothercommonmistakeistoassumethatapracticeisdiscouragedorprohibitedjustbecauseScrumdoesnotspecificallyrequireit.Forexample,ScrumdoesnotrequiretheProductOwnertosetalong-termstrategyforhisorherproduct;nordoesitrequireengineerstoseekadvicefrommoreexperiencedengineersaboutcomplextechnicalproblems.Scrumleavesittotheindividualsinvolvedtomaketherightdecision;andinmostcases,bothofthesepractices(alongwithmanyothers)arewelladvised.
SomethingelsetobewaryofismanagersimposingScrumontheirTeams;ScrumisaboutgivingaTeamspaceandtoolstomanageitself,andhavingthisdictatedfromaboveisnotarecipeforsuccess.AbetterapproachmightbeginwithaTeamlearningaboutScrumfromapeerormanager,gettingcomprehensivelyeducatedinprofessional training,andthenmakingadecisionasaTeamto followthepractices faithfully foradefinedperiod;attheendofthatperiod,theTeamwillevaluateitsexperience,anddecidewhethertocontinue.
ThegoodnewsisthatwhilethefirstSprintisusuallyverychallengingtotheTeam,thebenefitsofScrumtendtobevisiblebytheendofit,leadingmanynewScrumTeamstoexclaim:“Scrumishard,butitsureisawholelotbetterthanwhatweweredoingbefore!”
APPENDIX:TERMINOLOGYBurnDown ThetrendofworkremainingacrosstimeinaSprint,aRelease,oraProduct.ThesourceoftherawdataistheSprintBacklogandtheProductBacklog,withworkremainingtrackedontheverticalaxisandthetimeperiods(daysofaSprint,orSprints)trackedonthehorizontalaxis.
ChickenSomeonewhoisinterestedintheprojectbutdoesnothaveformalScrumresponsibilitiesandaccountabilities(Team,ProductOwner,ScrumMaster).
DailyScrum
AshortmeetinghelddailybyeachTeamduringwhichtheTeammembersinspecttheirwork,synchronizetheirworkandprogressandreportandimpedimentstotheScrumMasterforremoval.Follow-onmeetingstoadaptupcomingworktooptimizetheSprintmayoccuraftertheDailyScrummeetings.
Done
Completeasmutuallyagreedtobyallpartiesandthatconformstoanorganization’sstandards,conventions,andguidelines.Whensomethingisreportedas“done”attheSprintReviewmeeting,itmustconformtothisagreeddefinition.
ãJeffSutherland1993-2021 31
EstimatedWorkRemaining(SprintBacklogitems)
ThenumberofhoursthataTeammemberestimatesremaintobeworkedonanytask.ThisestimateisupdatedattheendofeverydaywhentheSprintBacklogtaskisworkedon.Theestimateisthetotalestimatedhoursremaining,regardlessofthenumberofpeoplethatperformthework.
IncrementProductfunctionalitythatisdevelopedbytheTeamduringeachSprintthatispotentiallyshippableorofusetotheProductOwner’sstakeholders.
IncrementofPotentiallyShippableProductFunctionalityAcompletesliceoftheoverallproductorsystemthatcouldbeusedbytheProductOwnerorstakeholdersiftheychosetoimplementit.
Sprint
Aniteration,oronerepeatingcycleofsimilarwork,thatproducesincrementofproductorsystem.Nolongerthanonemonthandusuallymorethanoneweek.Thedurationisfixedthroughouttheoverallworkandallteamsworkingonthesamesystemorproductusethesamelengthcycle.
PigSomeoneexercisingoneofthethreeScrumroles(Team,ProductOwner,ScrumMaster)whohasmadeacommitmentandhastheauthoritytofulfillit.
ProductBacklogAprioritizedlistofrequirementswithestimatedtimestoturnthemintocompletedproductfunctionality.EstimatesaremoreprecisethehigheranitemisintheProductBacklogpriority..Thelistemerges,changingasbusinessconditionsortechnologychanges.
ProductBacklogItemFunctionalrequirements,non-functionalrequirements,andissues,prioritizedinorderofimportancetothebusinessanddependenciesandestimated.TheprecisionoftheestimatedependsonthepriorityandgranularityoftheProductBacklogitem,withthehighestpriorityitemsthatmaybeselectedinthenextSprintbeingverygranularandprecise.
ProductOwnerThepersonresponsibleformanagingtheProductBacklogsoastomaximizethevalueoftheproject.TheProductOwnerisresponsibleforrepresentingtheinterestsofeveryonewithastakeintheprojectanditsresultingproduct.
ScrumNotanacronym,butmechanismsinthegameofrugbyforgettinganout-of-playballbackintoplay.
ScrumMasterThepersonresponsiblefortheScrumprocess,itscorrectimplementation,andthemaximizationofitsbenefits.
ãJeffSutherland1993-2021 32
SprintBacklogAlistoftasksthatdefinesaTeam’sworkforaSprint.ThelistemergesduringtheSprint.EachtaskidentifiesthoseresponsiblefordoingtheworkandtheestimatedamountofworkremainingonthetaskonanygivendayduringtheSprint.
SprintBacklogTaskOneofthetasksthattheTeamoraTeammemberdefinesasrequiredtoturncommittedProductBacklogitemsintosystemfunctionality.
SprintPlanningmeetingAone-daymeetingtimeboxedtoeighthours(forafourweekSprint)thatinitiateseverySprint.Themeetingisdividedintotwofour-hoursegments,eachalsotimeboxed..DuringthefirstfourhourstheProductOwnerpresentsthehighestpriorityProductBacklogtotheteam.TheTeamandProductOwnercollaboratetohelptheTeamdeterminehowmuchProductBacklogitcanturnintofunctionalityduringtheupcomingSprint.TheTeamcommitstothisattheendofthefirstfourhours.Duringthesecondfourhoursofthemeeting,theTeamplanshowitwillmeetthiscommitmentbydesigningandthendetailingitsworkasaplanintheSprintBacklog.
SprintRetrospectivemeetingAtimeboxedthree-hourmeetingfacilitatedbytheScrumMasteratwhichthecompleteTeamdiscussesthejust-concludedSprintanddetermineswhatcouldbechangedthatmightmakethenextSprintmoreenjoyableorproductive.
SprintReviewmeetingAtime-boxedfourhourmeetingattheendofeverySprintwheretheTeamcollaborateswiththeProductOwnerandstakeholdersonwhatjusthappenedintheSprint.ThisusuallystartswithademonstrationofcompletedProductBacklogitems,adiscussionofopportunities,constraintsandfindings,andadiscussionofwhatmightbethebestthingstodonext(potentiallyresultinginProductBacklogchanges).Onlycompletedproductfunctionalitycanbedemonstrated.
StakeholderSomeonewithaninterestintheoutcomeofaproject,eitherbecausetheyhavefundedit,willuseit,orwillbeaffectedbyit.
TeamAcross-functionalgroupofpeoplethatisresponsibleformanagingthemselvestodevelopanincrementofproducteverySprint.
TimeboxAperiodoftimethatcannotbeexceededandwithinwhichaneventormeetingoccurs.Forexample,aDailyScrummeetingistimeboxedatfifteenminutesandterminatesattheendoffifteenminutes,regardless.Formeetings,itmightlastshorter.ForSprints,itlastsexactlythatlength.
ãJeffSutherland1993-2021 33
ROLLINGOUTAGILEATALARGEENTERPRISEGabrielleBenefield
SeniorDirectorofAgileDevelopmentatYahoo!Inc.,2007
ABSTRACTYahoo!isa$50BcompanythathasoneofthelargestAgileimplementationsintheworld.TheadoptionofScrumandAgilepracticeshasbeensteadilygrowingoverthepasttwoyears,andnowencompassesmorethan150Yahoo!teamsandmorethan1500peopleintheUnitedStates,Europe,andAsia-Pacific.TheprojectsrangefromnewproductdevelopmentsuchasYahoo!Autostoheavy-dutyinfrastructureworkonYahoo!Mailwhichserves250millionuserseachmontharoundtheglobe.
INTRODUCTIONIn thehighly competitive Internet space, gettingproducts tomarketquicklywhilebeingboth flexible andadaptivetochangeiscritical.Yahoo!neededaprocessthatsupportedanInternetstart-upculturewithinthestructureofprovidingproductsandservicestomorethan500millionusersworldwide.
BACKGROUNDYahoo!wentfrombeingasmallstart-upandgrewtoalargeenterprisecompanyquickly.Thecompanystillseemslikealargestart-upwiththegoodandbadthatcomeswithit.Thethingspeoplelikedaboutbeingastart-upwasworkingcloselywithasmallsetofpeople,beingabletoquicklygetproductstomarket,thecodebase was relatively small and simple to work within and technical debt had not built up in it. Theinterdependenciesbetweenproductsaresmallandscalingforasmallsetofusersiseasytodealwithonthebackend and in the application layer. Standardization in brand, user interface and tools is fairlystraightforward.Ifyouneedsomethingyouusuallyknowwhotogotoandhowtofindthemtogetthingsdone.Itisalsoveryexcitingasyourampupquicklyandthemoneystartsrollingin.Asacompanygrowsitneedstodealwiththecomplexityofmanymovingpieces,morepeoplewhoyoudon’tknow,thelogisticsofseating,feedingandmakingthousandsofemployeeshappy.Youareunderintensepublicscrutiny,particularlyifyouareaPubliccompany. Legalconcernsheightenandifanythinggoeswrongtheeffectscanbemassiveandbroadreaching.Nolongercanyousimplylaunchaproduct,multiplestakeholdersneedtobeinvolvedinthedecisionmaking,multiplepropertiesmaybeaffectedifinterconnectedintheYahoo!portal.
Itisveryhardtotrackdowninformationasthesizeofthecompanygrows.Itappearstobeanaturaltrendforstart-upsthatgrowintolargecompaniestohireinpeoplewithbigcompanyexperience. Sometimesthesepeoplecanaddalotofvalue,sometimestheycanbringinoverlybureaucraticprocessesthatareatoddswiththe“just-get-things-done”start-upculturewhichdrewintheemployeesinthefirstplace.Peoplewhobuildsystemsfromthegrounduphavealotofpassionandownershipandaren’talwaysreadytoshareandthesystems,platformsandtoolsputinplacetoserviceonesetofneeds,thatofasmallcompanynolongerservicehundredsofthousands,sometimesmillionsofusers.Inanattempttomakesenseofscalingpainthekneejerkreactionistooftenputinprocessestomanageandcontrolsoftwaredevelopment.Oftentheselookexcellenton paper and in theory should work, unfortunately they can be at odds with the main ingredient inorganizations,people.
Yahoo!attemptedtocontrol thesoftwaredevelopmentprocessandreleasedawaterfallprocesscalledthe“ProductDevelopmentProcess”in2002.Theprocesswasrolledoutgloballyandtheuseofitwasmandatory.Unfortunatelyforthecreators(orperhapsfortunately)alotofteamssimplyignoredtheprocess,orwheretheycouldn’t ignoreitpaidlipserviceandmadeit lookliketheyadheredtothesteps. Theteamsthatdid
ãJeffSutherland1993-2021 34
followitfounditwasheavy,slowedthemdownandaddedlittlerealvalue.Managementfeltliketheywereincontrolbuttheteamsrebelled.Thereweresomegrassrootseffortsin2004totryoutsomeAgilepracticessuchasExtremeProgrammingandScrum. Thiswasledbytheteammembersor inonecasebyasmallercompany(StataLabs)thatYahoo!acquired.TobiasMayer,anengineeronateamstartedasmallgrassrootsmovementtospreadthewordandfoundhiswaytotheVPofProductDevelopmentatthetime,PeteDeemer.PetetohiscreditrealizedthattheheavyweightprocesshehadhelpedrolloutwasnotsucceedingashehadhopedandwascuriousaboutAgile.TobiasaskedEllenSalisbury,anengineeringleaderfromStataLabstogiveaninternaltechtalkonherexperiences.Thetalkwasinspirationalandpiquedpeoplesinterests.Inaluckyconfluenceofevents,PetehappenedtocontactJeffSutherlandandKenSchwaberwhenhewasintheBayArea(whereYahoo!’smainheadquartersarelocated)onthesamedayastheexecutiveteamhadanoffsitedinner.PeteinvitedJefftobeaguestspeakeratthedinnertosharehisexperienceswithScrum.Theexecutiveteamwasveryinspireduponhearinghisresearchthattheydecidedtosponsorapilotprogramonthespot.ThisledtotheofficialrolloutoftheScrumpilotprograminFebruaryof2005.
PeteandTobiasevangelizedthebenefitsofScrumamongsttheircontactsandmanagedtogetfourteamstovolunteertotryScrumfortwomonthsandparticipateinasurveytogatherdataabouttheirexperiences.TheteamscoveredabroadsetofproductsandservicesincludingthenewYahoo!Photos3.0,anewbackendforYahoo!Mail,internaltoolsformanagingsmallbusinesssitesandamediasitere-design.AsubsetofteamleadsweresenttoaScrumMasterclasswithKenSchwaber.Theteamsusedaverystandardout-of-the-boxScrumframeworktoaddressprioritizationconcerns,self-organizationandteamwork,greatercustomerinvolvementandincrementalproductreleases.AtthisstagelittleattentionwasputontechnicalpracticesasScrumwasseenasaneasyfirststeptotestthewaters.AttheendoftheirfirstmonthofusingScrumalltheteammembersandtheirmanagerswereinvitedtoparticipateinanonlinesurveytoanonymouslygathertheirfeedback.Theresponders’receivedacustomprintedScrumt-shirt forparticipating,whichalsoservedadualpurposetopromoteScrum.Theoverallresponseratewas71%(~85%forScrumPilotteammembers).ThequestionsaskedsoughttotrackandcollateinformationintheareasthatYahoo!wantedtoimproveuponwhichweremainlyqualitativeandfocusedonthehumanaspectsofsoftwaredevelopment.Thequestionsaskedpeopletorate their experiences against their previous process.
ãJeffSutherland1993-2021 35
ãJeffSutherland1993-2021 36
ãJeffSutherland1993-2021 37
thefeedbackwaspositive;theteamslikedtheprocessandexperience,andmanagementsawpositiveresults.Twoyearslaterwehaveoveronehundredteamsspreadaroundtheglobeandwecontinuetogrowrapidly.
ThecultureatYahooisverymuchlikealargestart-up.Thereisaconstantstreamofinnovativeideasfromeverydepartment,newproductfeaturesareconstantlyreleased,andthecompanystrivestobethefirsttomarketwithnewservices,whilemeetingtheneedsofourusers.Thefoundersstillworkatthecompany,andhaveremainedinvolvedinday-to-dayactivities.Theycontinuetoinstillinallemployeesaloveforthecultureandwork.ThecompanyiscommittedtopreservingthethingsthatmakeYahoo!greatwhileputtinginsomeprocessandpracticestohelpteamsdeliverbetterproductsfaster;thiswasthegreatestchallengewefacedwhenintroducingAgile.
WestartedwithScrum,implementingsmall,cross-functionalteamstoaddressorganizationalissues,highlightbusiness priorities and most importantly, create a collaborative environment. Next we added in AgileengineeringpracticesandLeanfundamentalstodelivergreaterbusinessvalue.
KICK-OFF
ãJeffSutherland1993-2021 38
To kick off the program, we focused on building excitement internally, andmotivating employees to getinvolved.WeinvitedguestspeakerslikeKenSchwaberandJeffSutherland,theinventorsofScrumtoaddressemployees,andhadanewemployeegiveatalkonherexperienceswithScrumatapreviouscompany.Wewere also fortunate to have the VP of Product Development evangelize the benefits of Scrum to the topexecutivesinthelanguagethattheycouldrelateto.
Oncewelaidthegroundworkforthepilotprogramweexperimentedwithanengagementmodelthatallowedustocoachmultipleteamsefficiently.Wherewehadthebandwidthwewouldworkcloselywithteamstogetthemupandrunning.
We found itworkedwell to startby spending some timepreparing theProductBacklogwith theProductOwner,thentrainthewholeteamtogether.AcoachwasassignedtoleadthefirstSprintplanningmeeting,stand-up,Sprintreviewandretrospective.ForthesecondsprintwelettheScrumMasterleadtheteamandweshadowedthem.Afterthat,dependingontheteam,wewoulddodrop-insandtrytokeepagoodconnectionwiththeProductOwnerandScrumMastertohelpguidethemthroughtheearlyiterations.Atthispoint,someteamswereoffandrunning,whileotherswithdifficultissuesneededsomeadditionalcoaching.Wealsofoundthatteamsunlearntthingsoverextendedperiodsoftimesowestayedincontactandcontinuallyre-engagedtoleadsomemasterretrospectivesandgivethemobjectiveadviceandcoaching.
Wefoundanevolutionaryapproachtobefarmoresuccessfulthanarevolutionaryonefor longtermgoodresults.Attimesweneededtopushtheenvelopeandtakerisksbuttryingtoriptheband-aidofftooquicklyisdangerousandcanleadtoultimatefailure.Forus,organizationalchangemeantgettingbuyinatalllevels,havingpeopleseeresultsandbeabletolearninasafeenvironment.
MANAGEMENTSUPPORTAgileisallaboutexperimentationandtheabilitytoinspectandadaptasanempiricalapproach.Attheendoftheday,nothingreallymatteredbutwhatemployeesactuallyexperiencedandallideaswereuselessunlessweexecutedonthemandcouldprovethattheyworked.Onethingthatwaskeytotheongoingfundingandsuccess of the programwas a quarterly survey. The surveywas distributed to all the teammembers andmanagers,andweusedthedatatohelpusimprove,workedwithteamsthatmostneededit,anddistributedthedatabacktomanagement(seeexamplesofthequestionsandresponsesattheendofthisreport).OvertimethesurveybecamelessusefulasawaytocompareScrumwiththeoldprocessaspeopleonlyhadScrumexperienceatthecompany.Wealsofoundthatafteracoupleofsurveyspeopledidn’twanttodoanymore,solongtermdatagatheringisnoteffectiveusingasurveyalone.
EMPLOYEESUPPORTWedecidedtokeeptheprogramvoluntaryandstilldo.Itwasagreedthatforaprocesstobetrulysuccessfulitneededtostandonitsownmerits.AlthoughAgilewasboughtinfromthetop-down,thefactthattheprogramwasnevermandatedmeantithadbottom-upsupport.
Whilewebuiltrelationshipsatalllevelsandmarketedthesuccessestothemanagementteam,therealdrivingforcewaslettingthewordspreadvirally.TheteamsusingScrumspreadthewordabouttheprocessandpeoplemoving throughout the company seeded new teams. We leveraged the experiences of the people in thetrenchestocreateaveryeffectivepromotionengine.
FEEDBACKWetriedtokeepalotoftransparencyaroundtheprocessandfeedbackwewerereceiving.Wegainedfarmorecredibilitybybeingopenandlettingpeopleknowthattheprocessisnotasilverbulletandacknowledgingthat
ãJeffSutherland1993-2021 39
changeishard.Bybeingupfrontwiththechallengeswewereabletoconfrontdifficultissuesandimprove.Wehadpaneldiscussionsand“techtalks”fromdifferentScrumteamstosharetheirexperiences,thechallengesoftransitioningtoAgileandhowtheydealtwithissues.AcoretenetofAgileisaroundtransparencyandwefeltthisshouldalsogoforthemethodologyitself.
Top-downmandatesthattriedtoenforceScrumpracticesinaby-the-bookfashionalwaysbackfiredforus,asdidteamsthatfollowedthepracticessozealouslytheylosttheforestforthetrees.Wedidhaveteamsthatsimplyweren’treadyorwillingtouseAgileandwehadtorespectthat;wedidn’twanttoforceourcoachestobetheprocesspoliceandbecomepartoftheproblem.
Wedidn’tgettoohunguponhavingtheperfecttrackingtools,trainingmaterials,coachingprogram,etc.inplace.Wemadealotofmistakesbutwealsoimprovedquicklybasedoniterativefeedback.Ourphilosophydictatedthatitwasbettertomaketheflightthanhaveourbagspackedperfectlyandstillbewaitingontheground.
ROLL-OUTTokickoffanenterpriseAgilerollout,wefounditreallyhelpedtohavepeoplewithrealexperienceinthefield.Thefoundationoftheoverallstrategywasbuiltonlessonslearnedandunderstandinghowtodealwithchange.Webuiltoutacentralizedteamofcoacheswhowerepassionateandgoodatbuildingrelationships.
TheteamhadamixtureofskillsincludingProductmanagement,QA,Design,ExtremeProgramming,ScrumandLean.Itwasusefultohavepeoplewithspecializationsinadditionaltogeneralizedcoachingsotheycouldbuildbridgesintodifferentfunctionalgroups.Personalitywasalsokey.Weneededpeoplewithpassionandenthusiasm,aswewereonlyasgoodas the relationshipswecould form.Havingpeoplewhowereoverlyzealousorabrasivewouldhavequicklyboughttheprogramtoahalt.Oneimportantaspectofhiringwastofindpeoplewithstrongskillsincollaborationandbuildingconsensus.
Next we found that the best Agile champions were the people already in the teams, from all levels anddisciplines.Thesepeopleknewthecontextandthechallengesoftheirparticularsituationandcouldadjusttheprocesstomeettheirteam’sneeds.Findinggoodpeoplewhoreallygetitandtrainingthemuptohelptheirownteamisoneofthedifferentiators,andistheonlywaytoscaleeffectivelyinalargeorganization.
CHALLENGESManagersoftenfeelleftoutwhentheteambecomesmoreself-organizedanddon’tknowhowtotransitionfromthetraditionalcommandandcontrolmodeltooneofastrategicandsupportiveleader.Theysometimeslashoutorsubverttheprocessoutoffear.Wherewecameacrosspeoplewhoareanti-Agile,wetriedtogetthem tounderstand their changedroleand togive themsomeresponsibility.Trainingandcoaching thesepeopleisworththeinvestment.Wealsodidhavetodealwiththerealitythatnoteveryoneiswillingorcanchangeandultimatelythenewenvironmentmaynolongerbeagoodfitforthemanymore.
Wewantedpeopletoparticipateinthesurveysbutafteracoupleofroundstheywereboredwithhavingtofilloutsurveyinformationalloveragain.Soweofferedfreet-shirts.Theseweren’tanyoldt-shirt,theywerecool.Wedidn’tevenbrandthemasthesheerambiguitywasveryappealingandpeoplewouldbecurioustoknowwhatthet-shirtsmeantandthiswouldopensomedoorsforus.WehadpeoplesigninguptotryScrumjusttogetat-shirt.
ãJeffSutherland1993-2021 40
AnotherchallengewehadandstillhavetosomeextentistokeeptothekeytenetsofScrumwhileadaptingtodifferentcontexts.Scrumprovidesanextremelyflexibleframeworkandhowyouapplyitisanopen-endedquestion.
WehaveaverystrongandbrilliantdesigngroupatYahoo!Ourproductsareheavilyconsumerfocusedsodesign is very important to us. TheDesigners initial reaction toAgilewas similar to theway engineeringarchitectsreactwhenfacedwiththeideathatyoudon’tdesigneverythingupfrontthatyouconstantlyre-factorandthatrequirementswillchange.
Workingtounderstandthechallengesandfindingcommongroundhelpedimprovethesituation.Therearesomethingsthatdofitwithdesignthinking.Designersdowanttoadaptandworkincrementally.Userstoriesthatarefocusedonthecustomerarealsowarmlygreeted.Leanthinkinginkeepingthefeaturestoaminimumanddoingthemwellalsostrikesapositivenote.
Wetriedtobeflexible,tolistentothedesignviewpointandtohelpthewholeteamfindawaytoworktogetherinawaythatmadesensetothem.Ifthewholeteamwasnotabletofindcommongroundthatworkedattheexpenseofkeymemberstheholisticteamdidnotsucceed.
WeallowedtheteamstofindawaytoworkwithintheAgileframeworkthatsuitedtheircontextandneeds,usingScrumasaflexibleframework.Wehaveteamsthatdooverlapworkwithineachiteration,wheresomedesign is done looking forward to the next iteration, some user testing of work completed in a previousiteration,andhandoffsduringtheiterationoccur.ThismaynotlooklikepureScrumbutitworksforteamsdevelopingconsumerfacingproductsandKenSchwaberhasalwaystriedtogetacrossthevaluesratherthantherulesofScrumsowetookthisverymuchtoheart.Ifwetriedtoenforceonlyworkingontasksforthecurrentiterationduringthecurrentiterationthedesignerswouldhavemutiniedandtheteamcollaborationwouldhavesuffered.Again,Scrumisadaptiveandifitworksforpeopletheykeepdoingit.
WehavefoundthoughthatthenumberonereasondesignerslikeAgileisthecollaborationaspect.Iftheteamspirit is strong and collaboration between team members is working they can overcome the logisticaldifficultiesastheyworktogether.
TRAININGItwasandstillisextremelychallengingtogetexecutivesandseniormanagerstoScrumtrainingduetotheirbusyschedules,butitisalsoinvaluableandworththeinvestment.OneGeneralManagertookatwodayScrumMasterclasswithhisteamandsaiditwasagreatexperience.Hegottohearthetoughchallengesandissuestheteamwerefacingwhiletheteamgotalotofinsightintothebusinesschallengesandvision.Thisestablishedahealthybaseforongoingtwo-wayconversations.Themanagerboughtalotofcredibilityforinvestingtimetositandlearninthesameroomaseveryoneelse.
Eventhoughwehadgreatinternalcoaches,wewereveryunderstaffed.WerealizedthathavinggreatpeoplelikeMikeCohn,KenSchwaber,JeffSutherland,MaryandTomPoppendieckwascrucialtogettingtheprogramoff the ground. It is worth spending themoney on consultants if you lack internal expertise as they canultimatelysavethecompanymoneyifyouapplythemwisely.
NEXTTIMETheAgileDevelopmentteamatYahoo!approachedthestrategywithpragmatismandadaptability,andhasexperiencedgreatsuccesswiththeprogram.Nevertheless,therearealwaysthingsyoucanchange,includingdedicating more resources and funding to the project, but until we could prove the process worked thebusinesswasnotgoingtoinvestalot.Thewholeprocesshasbeenandstillisallaboutlearningandadapting
ãJeffSutherland1993-2021 41
aswego.Thefailurespropelledustonewlevelsanditwasimportanttoallowteamstounderstandthatfailureisinitselfaneffectivelearningmechanism.
FUNDTHEINTERNALCOACHINGTEAMADEQUATELYItwouldhavebeengreattohavetheinternalcoachingteamstaffedadequatelysowecouldgetagoodscalingstrategyinplaceearlier.Iwasworkingbymyselfforaperiodoftime,andthegrouponlyhadtwofulltimecoachesconsistentlyforthefirstyear.Weputnewmeaningtotheterm“lean”.Ittookalongtimetogetmoreresourcesassignedtothecentralcoachingteamandthisonlycameafterfinancialanalysishelpedtoproveeach coach’s value (around 1.4 million dollars saved per year for each coach helping 10 teams be moreproductive).Thiswouldhavemadealotmoreteamsalotmoresuccessfulandwecouldhavescaledfasterandbetterwithadditionalresources.
ENCOURAGEDEEPERENGAGEMENTFROMCOACHINGSTAFFItwouldhavebeenpreferabletocoachteamsmoreintenselyratherthanbeingsobroadlyfocused.Duetoconstrainedresourcesandhugedemanditwasimpossibletoworkasdeeplywithteamsaswewouldhaveliked,andthisshowedveryclearlyinthesurveyresults.Theteamswithadequatecoachingshowedproductivityincreasesof2-3timesmorethanteamstryingtoworkbythemselves.
There are teams that kicked off by attending a public class and had no follow-up coaching, due to eitherbandwidthrestrictionsornotrealizingthevalueofit.WesometimesrunintotheseteamsorhearaboutthemthroughthegrapevineandfindouttheyarenotreallydoingScrumatallbutahybridthatallowsthemtocontinue their dysfunctional practiceswhile calling it Scrum.This is amajor problem sonowwhen teachclasseswestresstheimportanceofcoaching.
Itwouldhavealsobeengreat tohavesolidengineeringcoachesavailable fromdayone toworkwithnewteams, helping them set up build and test systems and introducing Agile engineering concepts. It is verychallengingtodeliverincrementalproductswithoutgoodengineeringdisciplineandthishasdefinitelyheldbacktheproductivityandqualityofmanyofourteams.
DEEPERMANAGEMENTINVOLVEMENTWecouldhavepushedhardertogetseniormanagementtoattendfocusedtraining.Havingonlyashallowunderstandinghasleadtomisunderstandingsthatcouldhavebeenavoidedifwehadbeenabletodomoreofthis.Again,inanidealworldwithmoreresourceswecouldhavedonemoretargetedtrainingformanagement,productmanagement,QAanddesigntobetterintegratethemintothecollectiveteamspirit.Wearecurrentlyworkingtowardsthis.
FOCUSONTHEMAJORITYItisagiventhatnoteveryonewillbehappyineverysituation.WefoundintheprocessofintroducingScrumtoYahoo!thatwedidhavepeoplewhowereverynegativetowardsourefforts.Fear,controlandpoliticsareconstantlychallengingandwehadtosimplyrealizeweweren’talwaysgoingtomakefriends.Peoplewillreacttothechangesandiftheydidn’t,youwouldprobablybetellingthemwhattheywantedtohear,notperhapswhattheyneedtohear.
CONCLUSIONAlthoughwehaveover100teamsatYahoo!,westillhavea longwaytogo.Somedays it feels likewearewinningandAgileisspreadingitsloveoverthewholecompany,otherdaysteamsreverttobadpracticesandnewblocksappearthatfeelimpossibletobreakthrough.SometeamsareveryAgile,othersdomini-waterfalls
ãJeffSutherland1993-2021 42
andcallitAgile.Changeisdifficult,andtochangeacompanyaslargeasYahoo!sometimesfeelsliketryingtosteertheTitanicwithasmallpaddle.Welearnedthatpatienceisimportant,asisrememberingthateventhesmallestofincrementalimprovementshaveamassivepayoffwhenyoudothematlargescale.
RESULTSFROMSCRUMThebenefitsofScrumreportedbyteamscomeinvariousaspectsoftheirexperience.Onceeachquarter,wesurveyedeveryoneatYahoo!usingScrum(includingProductOwners,TeamMembers,ScrumMasters,andthefunctionalmanagersofthoseindividuals)andaskthemtocompareScrumtotheapproachtheywereusingpreviously.Belowaresomeresultsfromourprevioussurveys:
•Productivity:68%ofrespondentsreportedScrumisbetterormuchbetter(4or5ona5-pointscale);5%reportedScrumisworseormuchworse(1or2ona5-pointscale);27%reportedScrumisaboutthesame(3ona5-pointscale).
•TeamMorale:52%ofrespondentsreportedScrumisbetterormuchbetter;9%reportedScrumisworseormuchworse;39%reportedScrumisaboutthesame.
•Adaptability:63%ofrespondentsreportedScrumisbetterormuchbetter;4%reportedScrumisworseormuchworse;33%reportedScrumisaboutthesame.
•Accountability:62%ofrespondentsreportedScrumisbetterormuchbetter;6%reportedScrumisworseormuchworse;32%reportedScrumisaboutthesame.
•CollaborationandCooperation:81%ofrespondentsreportedScrumisbetterormuchbetter;1%reportedScrumisworseormuchworse;18%reportedScrumisaboutthesame.
•Team productivity increased an average a 37% increase, based on the estimates of the ProductOwners.
•86%ofteam-membersstatedthattheywouldcontinueusingScrumifthedecisionweresolelyuptothem.
Contact:GabrielleBenefield([email protected])
ãJeffSutherland1993-2021 43
CAPTURINGEXTREMEBUSINESSVALUE:1000%ANNUALRETURNONINVESTMENTINSCRUMTRAINERSJeffSutherland,Ph.D.,PatientKeeper,Inc.,2007
ABSTRACT In2005,JeffSutherlandworkedtogetherwithPeterDeemeratYahoo!tobrieftheYahoo!seniormanagementteamonScrum.AfterseniormanagementmadethedecisiontomoveforwardwithScrum,aproductivityanalysisofrolloutofScrumataprevious largeenterprise(IDXSystems,nowGEHealthcare)wasusedtocalculateanannualROIof1000%onathreeyearrolloutofScrumatYahoo!AftertwoyearsofdeploymentandScrumrollouttoover100teamstherateofreturnatYahoo!forinvestmentineachinternalScrumtrainerwas$1.4basedontrainingof10teamsannuallypertrainer,orroughly1000%returnoninvestment.TeamscoachedbyaScrumtrainerachieved3-4timestheproductivitygainsofuncoachedteams[27].
INTRODUCTION TheinternalrateofreturnoninvestmentinScrumtrainingisquitehigh.Manycompanieshavedoubledtherateofsoftwareproductionontheaverageforallteamsmeasured.RecentlyaCMMILevel5companycutthecostsofsoftwareprojectsinhalfandreducedmeasureddefectsby40%whilestillmaintainingCMMILevel5complianceforallprojects[20].EventhebestcompanieswillradicallyimproveperformancebyintroducingScrumandsomewillachievefarmorethan1000%rateofreturnoninvestmentinScrumtraining.
ThispaperaddressestheROIonScrumtrainingfortheaveragelargecompanywiththousandsofemployeesandhundredsor thousandsofdevelopers.These companieshaveestablishedheavyweightprocessesovermanyyearsthatarebureaucraticandloadedwithwaste.Whileonthesurfaceitwouldappeareasytoprovidesubstantialgainsbyeliminatingthemostobvioussourcesofinefficiency,introducingaradicallynewprocesscompanywidecanbeslowandpainful.Scrumhasasystematiccontinuousqualityimprovementprocessthatidentifiesandprioritizescompanywideimpedimentstoprogress.
IDX SYSTEMS (NOW GE HEALTHCARE): SCALING SCRUM FOR THE FIRST TIME Duringthesummerof1996,IDXSystems(nowGEHealthcare)hiredJeffSutherlandasseniorVPofengineeringandproductdevelopment.IDXhadover4,000customersandwasoneofthelargestUShealthcaresoftwarecompanies,withhundredsofdevelopersworkingondozensofproducts.HerewasanopportunitytoextendScrumtolarge-scaledevelopment.
TheapproachatIDXwastoorganizetheentiredevelopmentgroupintoaninterlockingsetofScrums.Whilethiswasthefirstlargedevelopmentteamtotrythisapproach,thestrategyhasnowbeenexecutedmanytimesanddocumentedbyKenSchwaberin“ScrumintheEnterprise”[28].Everypartoftheorganizationwasteambased, includingthemanagementteam,whichincludedtwovicepresidents,aseniorarchitect,andseveraldirectors.Front-lineScrumsmetdaily.AScrumofScrums,whichincludedtheteamleadersofeachScruminaproductline,metweekly,ThemanagementScrummetmonthly.
ThekeylearningatIDXwasthatScrumscalestoanysize.Withdozensofteamsinoperation,themostdifficultproblemwasensuringthequalityoftheScrumprocessineachteam,particularlywhentheentireorganizationhadtolearnScrumallatonce.IDXwaslargeenoughtobringinproductivityexpertstomonitorthroughputoneveryproject.Whilemostteamswereonlyabletodoubletheindustryaverageinfunctionpointspermonthdelivered,severalteamsmovedintoahyperproductivestate,producingdeliverablefunctionalityatfourtofivetimestheindustryaverage.Theseteamsbecameshiningstarsintheorganizationandexamplesfortherestoftheorganizationtofollow.
ãJeffSutherland1993-2021 44
OneofthemostproductiveteamsatIDXwastheWebFrameworkteamthatbuiltawebfrontendinfrastructureforallproducts.TheinfrastructurewasdesignedtohostallIDXapplications,aswellasseamlesslyinteroperatewith end user or third party applications. TheWeb Framework was created by a distributed team withdevelopers in Boston, Seattle, and Vermont who met by teleconference in a daily Scrum meeting. Thegeographic transparency of thismodel produced the samehigh performance as co-located teams andhasbecomethesignatureofhyperproductivedistributed/outsourcedScrumsatXebiaintheNetherlands/IndiaandExigenServicesinUnitedStates/Russia[29].
ThequalityofsoftwareofmanyofthehyperproductiveScrumteamscanbeextraordinarilyhigh.TheIDXWebFrameworkwasfirstdeployedin1997andin2007wasselectedasthecorewebtechnologyforGEHealthcaresystems.However,veryfewoftheIDXsoftwareteamsachievedthehyperproductivestate.Ontheaverage,basedonfunctionpointanalysisbyCapersJonescompany,SoftwareProductivityResearch,IDXonlyachievedaverageproductivegainsof240%,primarilyduetolossofproductionbecauseoflargeScrumteams,upto15peopleinsize.Itiswellunderstoodtodaythattheselargeteamscausesignificantlossinproductivityandmakeitimpossibletoachievelinearscalability,oneofthekeyfeaturesofwell-executedScrumimplementationsinthebestScrumcompanies.
SUMMARY OF PRODUCTIVITY GAINS AT IDX ThebudgetoftheIDXdevelopmentorganizationwasalmost$50Mperyearandthiswassufficientsizetoadetailed analysis of baseline productivity before Scrum and productive gains generated by Scrum. Anindependentconsultingfirm,SoftwareProductivityResearch(SPR),washiredtodofunctionpointanalysisofeveryIDXsoftwareproduct.JeffSutherlandhadworkedwithCapersJones[30],thefounderofSPR,duringtheoriginalcreationofScrumandwantedtocomparetheproductivitygoalsdesignedintotheScrumprocesswithactual performance in the field. Some of the IDX productswere sized at over 12000 function points, theequivalent of over amillion lines of Java or C# code and the smaller productswere typically 5000-6000functionpoints. Applicationswere financial and clinical products for operatinghospitals and independentphysiciangroupsatthousandsofsites.ImplementationplatformsvariedfromMumpstoCoboltoJavaandthelatestMicrosofttoolsandlanguagesavailable.
Function points were chosen as an industry standard measure that was independent of the softwaredevelopment language and environment. This was to ensure realistic comparison of productivity acrosstechnologies and development teams, as well as comparability with external industry data. Externalprofessionalexpertswerehiredtocalculatefunctionpointsinordertoprovideresearchqualitydata.Functionpoints arenot easy calculatedby the averagedevelopment teamandnot recommendedas anoperationalstrategy.StorypointshaveemergedasindustrybestpracticeformeasuringAgiledevelopmentteamvelocity[31].Theyareeasilycalculatedandusefulforreleaseplanning.However,theyarenotcomparableacrossteamsor across companies so were not suitable for research data on the first deployment of Scrum in a largeenterprise.
Ateveryreleasepointforeveryproduct,thenumberoffunctionpointswasrecalculatedtoreflectthenewfeaturesbySoftwareProductivityResearchconsultants.Theincreaseinfunctionpointswasdividedbypersonmonths for the fully burdened development teams including design, coding, testing, administrative, andmanagementstaff.Initialvelocityofalldevelopmentteamswasindustryaverageat2-3functionpointsperstaffmonth.
Some teams accelerated into a hyperproductive state using Scrum, achieving 5-10 times industry averageperformance.Theseteamswereabout10%oftheorganizationandachievedanaverageproductivityincreaseof666%.Asmallnumberof teams(less than5%)experienced failures foreitherpersonnelor technology
ãJeffSutherland1993-2021 45
reasons.Occasionally,ateamwouldnotbeabletoworktogethereffectivelyandwasreorganizedordisbanded,usuallyduringthefirstSprint.Lessfrequently,ahighperformanceteamhadtakenacalculatedriskonnewtechnologies(alwaysapprovedbymanagement)andtechnicalfailureofsomeSprintswasanticipated(evenencouraged)inordertogainatechnologyleadinthemarket.Therewerenofailuresoflargeprojects,onlyfailuretodeliversoftwareinisolatedSprintsorashortseriesofSprints.Inthecaseofteamfailures,theteamswerealways reformed. In the caseof technology failures, effortswere redoubled in subsequentSprints toovercomeresearchanddevelopmentchallenges.
Theremaining85%ofteamsachievedanaveragevelocityof5-6functionpointsperstaffmonth,averaginga100%gain.Thenetproductivitygainofallteamscombinedwas240%.ThiswasviewedasafailuretoachieveToyotalevelperformance.However,itwasagoodfirststartforenterprisewidedeploymentofScrum.
It is important to note that these productivity gains were achieved at a sustainable pace with increasedemployeeretentionandenhancedabilitytohirethebestpeopleinthesoftwareindustryduetothehighqualityworkingenvironmentprovidedbyScrumfordevelopers.Thehyperproductiveteamswerealwaysthemostspirited teams who loved their jobs and worked closely together like a professional sports team.Hyperproductivity is not achieved by working harder, but only by working better through intensecommunication,mutualsupport,andan“effortless”skillthatmakeshardthingslookeasy.ThinkofMichaelJordangoingupforabasketballshot.Theteamhassethimupandtheshotoftenlookssosmoothandeasyitgeneratesexhilarationinboththeplayersandthespectators.
ãJeffSutherland1993-2021 46
CHAPTER2:THEFIRSTSCRUMScrum was derived from best practices in the Japanese auto and consumer products industry at EaselCorporationin1993.ThestoryofthefirstScrumwaspublishedbytheCutterAgileAdvisoryServiceoutlininglessonslearnedonthefirstScrum[22].
SeniormanagementsupportisveryhelpfulinimplementingScrumandatEasel,theCEOagreedtouseScrumforthefirsttimeonthemostcriticalprojectinthecompany.Scrumwasdesignedtotakethetoughestprojectandturnitintoasuccess.ItwouldneverhavebegunwithoutthesupportoftheEaselCEO.HowdoyousellScrumtomanagementwithoutanypastexperienceorreferencestorelyupon?
In1993, the1986paper in theHarvardBusinessReviewbyTakeuchi andNonaka [1] combinedwith theCoplien paper on the development of Quattro forWindows at Borland [2] triggered the first daily ScrummeetingsandthemonthlySprintcycle.JapanesebestpracticesinnewproductdevelopmentatHondaandFuji-XeroxremindedTakeuichiandNonakaoftheScrumformationinRugby.ItwouldtakemorethanthesepaperstoconvinceaCEOunderpressuretoapproveanewprocesshehadneverseenbefore.Whatwerethekeyarguments?
Today,thereareROIanalyses,experiencereports,successstories,andleanmanufacturingpracticesthathelpmakea compelling case for selecting Scrum.Toyotahas emergedas the leading Japanese exampleof leanproductdevelopment.ManypublicationsdocumentToyota’sprocesswhichachieves4timestheproductivityand12timesthequalityofatypicalU.S.competitor.ThisiswhatcanbeexpectedfromahighqualityScrumimplementationinsoftwaredevelopment.AsaresultOpenViewVenturesPartnersaskedseniormanagementteamsinalloftheirportfoliocompaniestostartlearningaboutScrumbyreadingthe“ToyotaWay”[14].
Scrumtrainingwassignificantlyupgradedin2016bythejointventureScrumIncJapanwhoarethetrainersforToyotaCorporateatToyotaCity.Thenewtrainingincludesfourcomponents:(1)theScrumGuide,(2)leantoolsandtechniques, (3)hyperproductivepatterns fromTheScrumBook:TheSpiritof theGame,andtheScrum@Scale Guide. ScrumInc Japan was founded by Scrum Inc and KDDI, one of the largest telecomcompaniesinJapanthatis20%ownedbyToyota.
Inaddition,therearetwoexcellentpublicationsformanagementofScrumcompaniesthathavebeenaddedtotheoriginalTakeuchiandNonakapaperintheHarvardBusinessReview–EmbracingAgile:Howtomastertheprocessthat’stransformingmanagementbyDarrellRigby,JeffSutherland,andHirotakaTakeuchiandAgileatScalebyDarrellRigby,JeffSutherland,andAndyNoble(May-June2016).
Agile at Scale is also featured inHBR's 10MustReadsonChangeManagement,Vol. 2 (withbonus article"Accelerate!"by JohnP.Kotter)byHarvard Business Review, John P. Kotter, Tim Brown, Roger L. Martin, Darrell K. Rigby
ãJeffSutherland1993-2021 47
AGILEDEVELOPMENT:LESSONSLEARNEDFROMTHEFIRSTSCRUMJeffSutherland,Ph.D.,PatientKeeper,Inc.,2004
INTRODUCTION ScrumforsoftwaredevelopmentteamswasdevelopedatEaselCorporationin1993,wherewebuiltthefirstobject-oriented design and analysis (OOAD) tool that incorporated round-trip engineering. In a Smalltalkdevelopmentenvironment,codewasauto-generatedfromagraphicdesigntoolandanychangestothecodefromtheSmalltalkintegrateddevelopmentenvironment(IDE)wereimmediatelyreflectedbackintodesign.
Sincetheproductwasdirectedtowardenterprisesoftwaredevelopment,wespentalotoftimeanalyzingbestpracticesinsoftwaredevelopmentmethodologies.
REVIEWING SOFTWARE DEVELOPMENT PROCESSES Werealizedweneededadevelopmentprocessthatfitanenhancedversionofrapidapplicationdevelopment,wherevisualizationofdesigncouldresultimmediatelyinworkingcode.Thisledtoanextensivereviewofboththeliteratureandtherealexperiencefromleadersofhundredsofsoftwaredevelopmentprojects.
There were some key factors that influenced the introduction of Scrum at Easel Corporation. “WickedProblems, Righteous Solutions” [32] reviewed the reasons why the waterfall approach to softwaredevelopmentdoesnotwork.
• Requirementsarenotfullyunderstoodbeforetheprojectbegins,
• Usersknowwhattheywantonlyaftertheyseeaninitialversionofthesoftware,
• Requirementschangeoftenduringthesoftwareconstructionprocess,
• Andnewtoolsandtechnologiesmakeimplementationstrategiesunpredictable.
DeGraceandStahlreviewed“All-at-Once”modelsofsoftwaredevelopmentthatuniquelyfitobject-orientedimplementationofsoftwareandhelpresolvethesechallenges.
“All-at-Once”modelsassumethatthecreationofsoftwareisdonebysimultaneouslyworkingonrequirements,analysis,design,coding,andtesting,thendeliveringtheentiresystemallatonce.Thesimplest“All-at-Once”modelisasinglesuper-programmercreatinganddeliveringanapplicationfrombeginningtoend.Allaspectsofthedevelopmentprocessresideinoneperson’shead.Thisisthefastestwaytodeliveraproductthathasgoodinternalarchitecturalconsistencyandisthe“hacker”modelofimplementation.Forexample,inaprojectbeforethefirstScrum,asingle individualspenttwoyearswritingeverylineofcodefortheMatisseobjectdatabaseused todrive$10Bnuclearreprocessingplantsworldwide.At less than50,000 linesofcode, thenuclearengineers said itwas the fastestandmost reliabledatabaseeverbenchmarked fornuclearplants.BrookshasdocumentedavariantofthisapproachcalledtheSurgicalTeam,whichIBMhasshowntobetheirmostproductivesoftwaredevelopmentprocess[33].
TheSurgeonorsuper-programmerapproachhasthefatalflawthatthereareatmostoneortwoindividualseveninalargecompanythatcanexecutethismodel.Forexample,ittookyearsforaleadingteamofdeveloperstounderstand the conceptual eleganceof theMatisseobject server technologyenough tomaintain it. Thesingle-programmermodeldoesnotscalewelltolargeprojects.
ãJeffSutherland1993-2021 48
Thenextlevelof“All-at-Once”developmentishandcuffingtwoprogrammerstogether,asinpairprogrammingin theeXtremeProgrammingparadigm[24].Here, twodevelopersworkingat thesame terminaldeliveracomponent of the system together. This has been shown todeliver better code (usability,maintainability,flexibility, extendibility) faster than two developersworking individually [34]. The challenge is achieve asimilarproductivityeffectwithmorethantwopeople.Whatisthebestwaytoworkwithmultipleteamsofpeopleonlargesoftwareprojects?
Our scalable, team-based “All-at-Once” model was motivated by the Japanese approach to new productdevelopment.Wewerealreadyusinganiterativeandincrementalapproachtobuildingsoftware[35].Itwasimplementedinslicesinwhichanentirepieceoffullyintegratedfunctionalityworkedattheendofaniteration.What intrigueduswasTakeuchiandNonaka’sdescriptionof the team-buildingprocess forsettingupandmanagingaScrum[1].Theideaofbuildingaself-empoweredteaminwhicheveryonehadtheglobalviewoftheproductonadailybasisseemedliketherightidea.Theapproachtomanagingtheteam,whichhadbeensosuccessful atHonda, Canon, and Fujitsu, also resonatedwith the systems thinking approach promoted byProfessorSengeatMIT[16].
We were prodded into setting up the first Scrum meeting after reading Coplien’s paper on Borland’sdevelopmentofQuattroProforWindows[2].TheQuattroteamdeliveredonemillionlinesofC++codein31monthswitha4-personstaffthatlatergrewto8.Thiswasabout1,000linesofdeliverablecodeperpersonperweek,themostproductivesoftwareprojecteverdocumented.Theteamattainedthislevelofproductivityby intensive interaction in daily meetings with project management, product management, developers,documenters,andqualityassurancestaff.
WHY THE EASEL CEO SUPPORTED THE FIRST SCRUM TheprimarydriverforbeginningthefirstScrumwasabsolutecommitmenttoadate,wherefailurewouldbreakthecompany.Wehadtoguaranteeddeliveryofaninnovativeproducttothemarketthatwouldachieverapidadoption.
MeetingwiththeCEO,IpointedoutthathehadbeengivenplansforyearsthatweresupportedbyGANTTcharts. He agreed no plan had ever delivered the required functionality on time. Many delays had beenextensiveandhurtthecompanyfinancially.Forecastedrevenueonamajornewproductupgradewasmillionsofdollarsamonthsoeverymonthlatecostthecompanymillionsinrevenue.Wecouldnotaffordlatedeliveryagain,asthecompanywouldoperateatalossforaquarterormoreanddamagetothestockpricewouldbesignificant.
Further, Ipointedoutthat inthepast,hehadnovisibilityonwherethesoftwarewas inthemiddleof theproject.HehadGANTTchartsandreportsthatlookedgoodonpaperbutneverdeliveredthesoftwareontime.Hehadneverseenapromiseddeliverydatemetandworse,herarelydiscoveredslippageuntilitwastoolatetoreforecastcompanyrevenue.
IsaidtomyCEOthat ifweadoptScrum,wesettheobjectivesatthebeginningofaSprint. It istheteam’sresponsibilitytofigureouthowtobestmeetthoseobjectives.DuringtheSprint,noonecanbothertheteammembers.AttheendofaSprint,Iadded,wewillhaveworkingcodethatcanbedemonstratedsohecouldseetheprogressbeingmade.YoucandecidetoshipanytimeordoanotherSprinttogetmorefunctionality.Visibleworkingcodewillgiveyoumoreconfidencethanextensivedocumentationwithnooperationalsystem.
ãJeffSutherland1993-2021 49
WecommittedtoafixeddatesixmonthsoutandplannedforsixmonthlySprints.TheCEOagreedtoproceedwiththefirstsoftwaredevelopmentScrum.Ittookhimabout60secondstodecide.Littledidheknowhowmuchofthefutureofglobalsoftwaredevelopmentrestedonthatdecision!
SCRUM BASICS ThefirstScrumstartedwithahalf-dayplanningsessionthatoutlinedthefeaturesetwewantedtoachieveinasix-monthperiod,andthenbroke it intosixpiecesthatwereachievable in30-daySprints. ThiswastheProductBacklog.ForthefirstSprint,theProductBacklogwastransformedintodevelopmenttasksthatcouldbedoneinlessthanonedayeach,thefirstSprintBacklog.
Shortdailymeetingswereessentialtodrivetheprojectwithcommonmindshare.ThethreeScrumquestionswereusedinthefirstSprint.Whatdidyoudoyesterday,whatwillyoudotoday,andwhatisgettinginyourway?DailymeetingsatEaselweredisciplinedinthewaythatwenowunderstandastheScrumpattern[26].This radically altered the nature of the software development process. It allowed sharing of the state ofsoftwarecomponentssothatdevelopmenttasks,thoughttotakedays,couldoftenbeaccomplishedinhoursusingsomeoneelse’scodeasastartingpoint.
OneofthemostinterestingeffectofScrumonEasel’sdevelopmentenvironmentwasanobserved“punctuatedequilibrium”effect.Thisoccursinbiologicalevolutionwhenaspeciesisstableforlongperiodsoftimeandthen undergoes a sudden jump in capability. During the long period of apparent stability, many internalchangesintheorganismarereconfiguredthatcannotbeobservedexternally.Whenallpiecesareinplacetoallowasignificantjumpinfunctionality,externalchangeoccurssuddenly.Afullyintegratedcomponentdesignenvironmentleadstounexpected,rapidevolutionofasoftwaresystemwithemergent,adaptivepropertiesresemblingtheprocessofpunctuatedequilibriumobservedinbiologicalspecies.SuddenleapsinfunctionalityresultedinearlierthanexpecteddeliveryofsoftwareinthefirstScrum[36].
Thisaspectofself-organization,thecreatorsofScrumnowunderstandasatypeofSetBasedEngineeringthatispracticedatToyota.Differentdeveloperswereworkingonmanycomponentsofthesystemandtryingtoevolvethemasfastaspossible.DecisiononhowtoimplementataskfromaSprintBacklogwasdelayeduntilthe last possible moment. The most evolved component for the task was selected to absorb the newfunctionality.
Byhavingeverymemberoftheteamseeeverydaywhateveryotherteammemberwasdoing,webegantogetcomments fromonedeveloper that ifhe changeda few linesof code,he couldeliminatedaysofwork foranotherdeveloper.Thiseffectwassodramaticthattheprojectacceleratedtothepointatwhichithadtobesloweddownbyoutnumberingdeveloperswithdocumentationandtestingengineers.ThishyperproductivestatewasseeninamanysubsequentScrums,althoughneverasdramaticasthefirstoneatEasel.Itwasacombinationof(1)theskilloftheteam,(2)theflexibilityofaSmalltalkdevelopmentenvironment,and(3)thewayweapproachedproductionprototypesthatrapidlyevolvedintoadeliverableproduct.
Forexample,akeytoenteringahyperproductivestatewasnotjusttheScrumorganizationalpattern.Wedidconstantcomponenttestingoftopicareas,integrationofpackages,refactoringofselectedpartsofthesystem,andmultiplebuildsperday.TheseactivitieshavebecomekeyfeaturesofeXtremeProgramming[37].
AddingtheSetBasedEngineeringpracticeisnowviewedasthe“secretsauce”thatturbochargedtheprocess.SowearestilllearninglessonsfromthefirstScrum.ThemagichappenswhentheScrumprocesscombineswithgoodengineeringpracticesandasophisticatedapproachtoproductevolution.SetBasedEngineeringcausedpunctuatedequilibriumanduncontrollablyfastproliferationoffunctionality.
ãJeffSutherland1993-2021 50
WeheldademoeveryFridayduringthefirstScrumandbroughtdevelopmentexpertsfromothercompaniesintolookattheproduct.Asaresult,ourdevelopershadtododemostopeersinothercompanies.ThiswasoneofthemostpowerfulacceleratorsIhaveseeninsoftwaredevelopment.Theoutsideexpertswouldsay,"Thatsucks, lookatBorland'sProductXtoseehowitshouldbedone."Or"Howcouldyoupossiblehaveadumbbuglikethat?"
Thenextweek,everythingwouldbefixed!Thedevelopersrefusedtobeembarrassedagaininfrontoftheirpeers.ThetotaltransparencyencouragedbyScrumwasextendedoutsidethecompanyandMITandRoute128leadengineersself-organizedtothrowtheScrumintooverdrive.ThiswasverychallengingtotheScrumteam.Everyweektheyfelttheywerenotgoodenoughandweredepressed.Ikeptremindingthemthattobeworldclass,wehadtorepeatedlyfacedefeatandtriumphoverit.WenowunderstandfromthePresidentofToyotathatthisrepeatedfailure,alongwithinspectingandadapting, isa fundamentalpracticethatallowspersonsandteamstomovetoahigherlevelofpractice.
Attheendofeachmonth,theCEOgothisdemo.Hecouldusethesoftwarehimselfandseeitevolving.Wethengavethesoftwaretotheconsultinggrouptouseinprototypingconsultingprojects.ThisgaveusanincredibleamountoffeedbacktoincorporateintotheProductBacklog,alistoffeaturesthataredesirabletohaveinthesoftware. At the beginning of each Sprint, the Product Backlog is reprioritized before transformation intodevelopmenttasks.TheScrumadaptabilitytochangeallowedtheCEOandtheProductOwnertosteerproductdevelopmentmoreeffectivelythanotherprojectmanagementtechniques.
SCRUM RESULTS TheCEOsawsignificant,step-by-stepprogressineachincrementandagreedthesoftwarewasreadytoshipin the fifth increment. Ithadmore functionality thanexpected in someareasand less inothers.The sixthincrementwasprimarily apackaging increment.We shippedon theday theproductwas scheduled tobeshipped.
Wegaveamoney-backguarantee that thisnewsoftwarewoulddoubledeveloperproductivity in the firstmonthofuse.ItsoldwelluntiltheSmalltalkmarketstartedtohitthewallinthemid-1990s,andbecameamodelforRationalRosedevelopment.ThefirstScrumMaster,JohnScumniotaleswentontoleadtheRationalRosedevelopmentteamafewyearslater.
Everyoneagreedthat(1)Scrumcouldmeetadeadline,(2)morefunctionalitywasachievedthanexpected,and(3)therewouldneverbeareturntoawaterfall-typementalitybecause(1)waterfallcouldnotpredict,(2)itcouldnotdeliverontime,(3)itproducedlessfunctionalityperdeveloperunitoftime,and(4)usersatisfactionwasterriblewhentheproductwasdelivered,sincewaterfallapproachesdidnotlendthemselvestocustomerinvolvementoralterationofspecificationsrequiredbyrapidlychangingmarketconditions.
Overthelastdecade,Scrumhasemergedfromhumblebeginningstoamovementinvolvingtensofthousandsofprojectsinhundredsoftheleadingsoftwaredevelopmentcompaniesworldwide.TheprocessmodelusedinthefirstScrum1993isessentiallythesameastaughtinScrumMastercoursesin2007.
ãJeffSutherland1993-2021 51
THEFIRSTSCRUM:HOWSCRUMPROVIDESENERGY,FOCUS,CLARITY,ANDTRANSPARENCYTOPROJECTTEAMSDEVELOPINGCOMPLEXSYSTEMSINTERVIEWWITHDR.JEFFSUTHERLAND,CTOOFPATIENTKEEPER,INC.EWORK-OUTBUSINESSPROCESSES,MAY,2006“…THEPRIMARYDRIVERFORBEGINNINGTHEFIRSTSCRUMWASABSOLUTECOMMITMENTTOADATE,WHEREFAILUREWOULDBREAKTHECOMPANY.THETASK:GUARANTEEDDELIVERYOFANINNOVATIVEPRODUCTTOTHEMARKETTHATWOULDACHIEVERAPIDADOPTION.“INAMEETINGWITHTHECEO,INOTEDTHATFORYEARSHEHADRECEIVEDPROJECTPLANSTHATWERESUPPORTEDBYGANTTCHARTS.THECEOAGREEDTHATNOPLANHADEVERDELIVEREDTHEREQUIREDFUNCTIONALITYONTIME.MANYDELAYSHADBEENEXTENSIVEANDHURTTHECOMPANY
FINANCIALLY.“FORCASTEDREVENUEONAMAJORNEWPRODUCTUPGRADEWASMILLIONSOFDOLLARSAMONTH,SOEVERYMONTHTHATAPROJECTWASLATECOSTTHECOMPANYMILLIONSINREVENUE.ASTHECOMPANYWOULDOPERATEDATALOSSFORAQUARTERORMOREANDDAMAGETOTHESTOCKPRICE
WOULDBESIGNIFICANT,WECOULDNOTAFFORDTOREPEATTHISCYCLE.“FURTHER,IPOINTEDOUTTHATTHECEOHADNOVIEWOFTHESTATUSOFTHESOFTWAREBYTHEMIDDLEOFTHEPROJECT.HEHADGANTTCHARTSANDREPORTSTHATLOOKEDSOLIDONPAPERBUTFAILEDTODELIVERTHESOFTWAREONTIME.HEHADNEVERSEENAPROMISEDDELIVERYDATEMET,ANDWORSE,HERARELYDISCOVEREDSLIPPAGEUNTILITWASTOOLATETOREFORECASTCOMPANYREVENUE.“ITOLDTHECEOTHATINADOPTINGSCRUM,WESETTHEOBJECTIVESATTHEBEGINNINGOFWHATSCRUMREFERSTOASASPRINT.ITISTHETEAMSRESPONSIBILITYTODETERMINEHOWTOBESTMEETTHOSEOBJECTIVES.DURINGTHESPRINT,NOONECANBOTHERTEAMMEMBERSWITHREQUESTS.ATTHEENDOFASPRINT,IADDED,WORKINGCODETHATWILLBEDEMONSTRATED,SOYOUCANSEETHEPROGRESSMADE.YOUCANDECIDETOSHIPANYTIMEORDOANOTHERSPRINTTOGETMOREFUNCTIONALITY.VISIBLEWORKINGCODEPROVIDESMORECONFIDENCETHANEXTENSIVEDOCUMENTATIONWITHNOOPERATIONALSYSTEM.“INTHECASEOFTHISPROJECT,THEDATEWASSIXMONTHSOUT,ANDWEESTABLISHEDSIXSPRINTS.THECEOAGREEDTOPROCEEDWITHTHEFIRSTSOFTWAREDEVELOPMENTSCRUM.“THEFIRSTSCRUMSTARTEDWITHAHALFDAYPLANNINGSESSIONTHATOUTLINEDTHEFEATURESETWEWANTEDTOACHIEVEINASIXMONTHPERIOD.WETHENBROKEITINTOSIXPIECESWHICHWERE
ACHIEVABLEIN30DAYSPRINTS.THISWASTHEPRODUCTBACKLOG.FORTHEFIRSTSPRINT,THE
ãJeffSutherland1993-2021 52
PRODUCTBACKLOGWASTRANSFORMEDINTODEVELOPMENTTASKSTHATCOULDBEDONEINLESS
THANADAY.“DAILYMEETINGSALLOWEDEVERYONEONTHEPROJECTTEAMTOSEETHESTATUSOFALLASPECTSOFTHEPROJECTINREALTIME.THISALLOWEDTHECOLLECTIVENEURALNETWORKSOFTHETEAM'SMINDTOFINE-TUNEORREDIRECTEFFORTSONADAILYBASISTOMAXIMIZETHROUGHPUT.THERESULTWASRADICALALTERATIONOFTHESOFTWAREDEVELOPMENTPROCESSBYALLOWINGSHARING
OFSOFTWARERESOURCES.DEVELOPMENTTASKSTHOUGHTTOTAKEDAYSCOULDOFTENBEACCOMPLISHEDINHOURSUSINGSOMEONEELSE'SCODEASASTARTINGPOINT.…“THEMEETINGSWEREKEPTSHORT,TYPICALLYUNDER30MINUTESANDDISCUSSIONWASRESTRICTEDTOTHETHREESCRUMQUESTIONS:WHATDIDYOUDOYESTERDAY?WHATWILLYOUDOTODAY?WHATOBSTACLESGOTINYOURWAY?“BYHAVINGEVERYMEMBEROFTHETEAMSEEEVERYDAYWHATEVERYOTHERTEAMMEMBERWASDOING,WECOULDMAKEPROGRESSBYIDENTIFYINGWORKTHATCOULDBEIMPROVEDBYOTHERS.WE
RECEIVEDCOMMENTSFROMONEDEVELOPER,FOREXAMPLE,THATIFHECHANGEDAFEWLINESOFCODE,HECOULDELIMINATEDAYSOFWORKFORANOTHERDEVELOPER.THISEFFECTWASSODRAMATICTHATTHEPROJECTACCELERATEDTOTHEPOINTATWHICHITHADTOBESLOWEDDOWNBY
OUTNUMBERINGDEVELOPERSWITHDOCUMENTATIONANDTESTINGENGINEERS.THISHYPERPRODUCTIVESTATEWASSEENINSEVERALSUBSEQUENTSCRUMS,ALTHOUGHNEVERASDRAMATICALLYASTHEFIRSTATEASEL.…“ATTHEENDOFEACHMONTH,THECEOGOTADEMO.HECOULDUSETHESOFTWAREHIMSELFANDSEEITWORK.WETHENGAVETHESOFTWARETOTHECONSULTINGGROUPTOUSEINPROTOTYPING
CONSULTINGPROJECTS.THISPROVIDEDANINCREDIBLEAMOUNTOFFEEDBACKTOINCORPORATEINTOTHESCRUMPRODUCTBACKLOG:ALISTOFDESIRABLEFEATURESTOINCLUDEINTHESOFTWARE.ATTHEBEGINNINGOFEACHSPRINT,PRODUCTBACKLOGISREPRIORITIZEDBEFORETRANSFORMATIONINTODEVELOPMENTTASKS.THESCRUMADAPTABILITYTOCHANGEENABLEDTHECEOTOSTEERPRODUCTDEVELOPMENTMOREEFFECTIVELYTHANOTHERPROJECTMANAGEMENTTECHNIQUES.“THECEOSAWSIGNIFICANT,STEPBYSTEPPROGRESSINEACHINCREMENTANDHEAGREEDTHATTHEPRODUCTWASREADYTOSHIPINTHEFIFTHINCREMENT.ITHADMOREFUNCTIONALITYTHANEXPECTEDINSOMEAREASANDLESSINOTHERS.WESHIPPEDONTHEDAYITWASSCHEDULEDTOBE
SHIPPED.
ãJeffSutherland1993-2021 53
“EVERYONEAGREEDTHATFIRST,SCRUMCOULDMEETADEADLINE;SECOND,MOREFUNCTIONALITYWASACHIEVEDTHANEXPECTED;ANDTHIRD,THEREWOULDNEVERBEARETURNTO[THEOLD]MENTALITY“OVERTHEPASTDECADE,SCRUMHASEMERGEDFROMHUMBLEBEGINNINGSTOAMOVEMENTINVOLVINGTENSOFTHOUSANDSOFPROJECTSINHUNDREDSOFTHELEADINGSOFTWARE
DEVELOPMENTCOMPANIESWORLDWIDE.PROPERLYIMPLEMENTED,SCRUMREPRESENTSBESTBUSINESSPRACTICEINSOMEOFTHEWORLD'SLEADINGCORPORATIONS.“ITALLOWSTEAMSTOOPERATECLOSETOTHE'EDGEOFCHAOS'TOFOSTERRAPIDSYSTEMEVOLUTION,ENFORCINGASIMPLESETOFRULESFORSELF-ORGANIZATIONOFSOFTWARETEAMSTOPRODUCESYSTEMSWITHEVOLVINGARCHITECTURES,ALIGNINGINDIVIDUALANDORGANIZATIONOBJECTIVES,CREATINGACULTUREDRIVENBYPERFORMANCE,SUPPORTINGSHAREHOLDERVALUECREATION,ACHIEVINGSTABLEANDCONSISTENTCOMMUNICATIONOFPERFORMANCEATALLLEVELS,ANDENHANCINGINDIVIDUALDEVELOPMENTANDQUALITYOFLIFE.”
ãJeffSutherland1993-2021 54
CHAPTER3:WHATISSCRUM?THEFIRSTPAPERSONTHESCRUMDEVELOPMENTPROCESS
Note:BestpracticesinScrumhaveevolvedovertheyearsandmaynotreflectedinthisearlypaper.Forexample, at PatientKeeper, there is no closure phase to a project. All user acceptance testing,documentationandtrainingisdoneaspartofaSprintandsoftwaregoesintoproductionatmultiplelargeenterprises.ThismeanstheSprintdemoinadvancedScrumimplementationsisalivesystemwiththousandsofusersattheendofeverySprint.
ManyofthecoreprinciplestaughtintheScrumMastertrainingcoursearetheonescreatedbythefirstScrumteam at Easel Corporation in 1993. Sprints, monthly iterations, daily meetings with three questions,impediments, and backlogs. The first Scrum also implemented all the eXtreme Programming engineeringpracticesinsomeformtwoyearsbeforeKentBeckcodifiedXP.
From1995-2000,JeffSutherlandchairedaseriesofworkshopsattheannualOOPSLAConferenceonBusinessObject Design and Implementation [38]. Ken Schwaber agreed to write the first paper on Scrum for theOOPSLA’95Workshop[39].HeobservedthefirstScruminactionandlaidoutthefundamentalprinciplesofoperation.OveradecadelaterthispaperisstilloneofthemostwidelyreadpapersfromOOPSLAConferences.Itcontinuestogetmorehitsthananyotherpaperathttp://jeffsutherland.com/Scrum.
Theconceptsofcomplexitytheoryareintroducedalongwiththeimportantdistinctionbetweenempiricalandpredictiveprocesses.Businessenterprisesarecomplexadaptivesystemsandthesoftwarethatrunsthemisrapidlybecomingequallycomplex.Softwaresystemsevolveovertimelikebiologicalsystems.Conceptsfromartificial life [6] are informative as systems that are flexible evolve faster as flexibility increasesup to theboundaryofchaos.UsingempiricalprocesscontroltopreventchaoticbehaviorwasshowntobetheessenceofScruminthispaper.
MikeBeedle[26]ledtheScrumprocessthroughelaborationasanorganizationalpatternsupportedbyKenSchwaber,JeffSutherland,andothers(seeAppendixI).FurtherinformationontheessenceofScrumcanbefoundinAgileDevelopmentwithScrumbyKenSchwaberandMikeBeedle[18]withcontributionsfromJeffSutherlandthatcanbefoundinapaperlaterinthisvolume–AgileCanScale:InventingandReinventingScruminFiveCompanies[40].
Over the last13years, Jeff Sutherlandhasused five companies as research laboratories for Scrum (Easel,VMARK,Individual,IDX,PatientKeeper)andKenSchwaberhasworkedasaconsultantinthelastthreeofthesecompanieswheretheyhavetestedandrefinedScrumtogether.OfcontinuedinterestiswhyafewScrumteamsenterthehyperproductivestatelikethefirstScrum.ThisisclearlyaffectedbythestagesofmaturityofScrumimplementations[41]alongwiththestructureofdeploymentofScrumteams[29].
ãJeffSutherland1993-2021 55
SCRUMDEVELOPMENTPROCESS
KenSchwaber
AdvancedDevelopmentMethods,Inc.,1995
_______________________________________________________________________
ABSTRACT
Thestated,acceptedphilosophyforsystemsdevelopmentisthatthedevelopmentprocessisawellunderstoodapproach that can be planned, estimated, and successfully completed. This has proven incorrect in practice.SCRUMassumesthatthesystemsdevelopmentprocessisanunpredictable,complicatedprocessthatcanonlyberoughlydescribedasanoverallprogression.SCRUMdefinesthesystemsdevelopmentprocessasa loosesetofactivitiesthatcombinesknown,workabletoolsandtechniqueswiththebestthatadevelopmentteamcandevisetobuildsystems.Sincetheseactivitiesareloose,controlstomanagetheprocessandinherentriskareused.SCRUMisanenhancementofthecommonlyusediterative/incrementalobject-orienteddevelopmentcycle.
KEYWORDS:ScrumSEICapability-Maturity-ModelProcessEmpirical
________________________________________________________________________
1.INTRODUCTION
Inthispaperweintroduceadevelopmentprocess,Scrum,thattreatsmajorportionsofsystemsdevelopmentasacontrolledblackbox.Werelatethistocomplexitytheorytoshowwhythisapproachincreasesflexibilityandabilitytodealwithcomplexity,andproducesasystemthatisresponsivetobothinitialandadditionallyoccurringrequirements.
Numerousapproachestoimprovingthesystemsdevelopmentprocesshavebeentried.Eachhasbeentoutedasproviding“significantproductivityimprovements.”Allhavefailedtoproducedramaticimprovements[42].AsGradyBoochnoted,“Weoftencallthisconditionthesoftwarecrisis,butfrankly,amaladythathascarriedonthislongmustbecallednormal.”[43]
ãJeffSutherland1993-2021 56
Concepts from industrial process control are applied to the field of systems development in this paper.Industrialprocesscontroldefinesprocessesaseither“theoretical”(fullydefined)or“empirical”(blackbox).Whenablackboxprocess is treatedasa fullydefinedprocess,unpredictable resultsoccur [44].A furthertreatmentofthisisprovidedinAppendix1.
Asignificantnumberofsystemsdevelopmentprocessesarenotcompletelydefined,butaretreatedasthoughtheyare.Unpredictabilitywithoutcontrol results. TheScrumapproach treats thesesystemsdevelopmentprocessesasacontrolledblackbox.
VariantsoftheScrumapproachfornewproductdevelopmentwithhighperformancesmallteamswasfirstobserved by Takeuchi andNonaka [1]3at Fuji-Xerox, Canon, Honda, NEC, Epson, Brother, 3M, Xerox, andHewlett-Packard.AsimilarapproachappliedtosoftwaredevelopmentatBorlandwasobservedbyCoplien[2]tobethehighestproductivityC++developmentprojecteverdocumented.Morerecently,arefinedapproachtotheSCRUMprocesshasbeenappliedbySutherland[45]toSmalltalkdevelopmentandSchwaber[46]toDelphidevelopment.
TheScrumapproachisusedatleadingedgesoftwarecompanieswithsignificantsuccess.WebelieveScrummaybeappropriateforothersoftwaredevelopmentorganizationstorealizetheexpectedbenefitsfromObjectOrientedtechniquesandtools[47].
2.OVERVIEW
Ournewapproachtosystemsdevelopmentisbasedonbothdefinedandblackboxprocessmanagement.WecalltheapproachtheScrummethodology(seeTakeuchiandNonaka,1986),aftertheScruminRugby--atightformationofforwardswhobindtogetherinspecificpositionswhenaScrumdowniscalled.
Aswillbediscussedlater,Scrumisanenhancementoftheiterativeandincrementalapproachtodeliveringobject-orientedsoftwareinitiallydocumentedbyPittman[48]andlaterexpandeduponbyBooch[49].ItmayusethesamerolesforprojectstaffasoutlinedbyGraham[50],forexample,butitorganizesandmanagestheteamprocessinanewway.
Scrumisamanagement,enhancement,andmaintenancemethodologyforanexistingsystemorproductionprototype. It assumes existing design and code which is virtually always the case in object-orienteddevelopmentduetothepresenceofclass libraries.Scrumwilladdresstotallyneworre-engineeredlegacysystemsdevelopmenteffortsatalaterdate.
Softwareproductreleasesareplannedbasedonthefollowingvariables:
ãJeffSutherland1993-2021 57
• Customerrequirements-howthecurrentsystemneedsenhancing.
• Timepressure-whattimeframeisrequiredtogainacompetitiveadvantage.
• Competition-whatisthecompetitionupto,andwhatisrequiredtobestthem.
• Quality-whatistherequiredquality,giventheabovevariables.
• Vision-whatchangesarerequiredatthisstagetofulfillthesystemvision.
• Resource-whatstaffandfundingareavailable.
Thesevariablesformtheinitialplanforasoftwareenhancementproject.However,thesevariablesalsochangeduringtheproject.Asuccessfuldevelopmentmethodologymusttakethesevariablesandtheirevolutionarynatureintoaccount.
3.CURRENTDEVELOPMENTSITUATION
Systemsaredevelopedinahighlycomplicatedenvironment.Thecomplexityisbothwithinthedevelopmentenvironmentandthetargetenvironment.Forexample,whentheairtrafficcontrolsystemdevelopmentwasinitiated,three-tierclientserversystemsandairlinederegulationdidnothavetobeconsidered.Yet,theseenvironmentalandtechnicalchangesoccurredduringtheprojectandhadtobetakenintoaccountwithinthesystembeingbuilt.
Environmentalvariablesinclude:
• Availability of skilled professionals - the newer the technology, tools, methods, and domain, thesmaller the pool of skilled professionals.
• Stabilityof implementation technology - thenewer the technology, the lower thestabilityand thegreater the need to balance the technology with other technologies and manual procedures.
• Stabilityandpoweroftools-thenewerandmorepowerfulthedevelopmenttool,thesmallerthepoolof skilled professionals and the more unstable the tool functionality.
ãJeffSutherland1993-2021 58
• Effectivenessofmethods-whatmodeling,testing,versioncontrol,anddesignmethodsaregoingtobe used, and how effective, efficient, and proven are they.
• Domainexpertise-areskilledprofessionalsavailableinthevariousdomains,includingbusinessandtechnology.
• Newfeatures-whatentirelynewfeaturesaregoingtobeadded,andtowhatdegreewillthesefitwithcurrent functionality.
• Methodology - does the overall approach to developing systems and using the selectedmethodspromote flexibility, or is this a rigid, detailed approach that restricts flexibility.
• Competition - what will the competition do during the project? What new functionality will beannounced or released.
• Time/Funding - how much time is available initially and as the project progresses? How muchdevelopment funding is available.
• Othervariables-anyotherfactorsthatmustberespondedtoduringtheprojecttoensurethesuccessoftheresulting,deliveredsystem,suchasreorganizations.
Theoverallcomplexityisafunctionofthesevariables:
complexity=f(developmentenvironmentvariables+targetenvironmentvariables)
wherethesevariablesmayanddochangeduringthecourseoftheproject.
As the complexity of the project increases, the greater the need for controls, particularly the ongoingassessmentandresponsetorisk.
Attemptstomodelthisdevelopmentprocesshaveencounteredthefollowingproblems:
ãJeffSutherland1993-2021 59
• Manyofthedevelopmentprocessesareuncontrolled.Theinputsandoutputsareeitherunknownorloosely defined, the transformation process lacks necessary precision, and quality control is notdefined.Testingprocessesareanexample.
• Anunknownnumberofdevelopmentprocessesthatbridgeknownbutuncontrolledprocessesareunidentified.Detailedprocessestoensurethatalogicalmodelcontainsadequatecontenttoleadtoasuccessfulphysicalmodelareonesuchprocess.
• Environmental input (requirements) can only be taken into consideration at the beginning of theprocess.Complexchangemanagementproceduresarerequiredthereafter.
Attemptstoimposeamicro,ordetailed,methodologymodelonthedevelopmentprocesshavenotworkedbecausethedevelopmentprocessisstillnotcompletelydefined.Actingasthoughthedevelopmentprocessisdefinedandpredictable,resultsinbeingunpreparedfortheunpredictableresults.
Although the development process is incompletely defined and dynamic, numerous organizations havedevelopeddetaileddevelopmentmethodologiesthatincludecurrentdevelopmentmethods(structured,OO,etc.).TheWaterfallmethodologywasoneofthefirstsuchdefinedsystemdevelopmentprocesses.ApictureoftheWaterfallmethodologyisshowninFigure1.
Figure1:WaterfallMethodology
Althoughthewaterfallapproachmandatestheuseofundefinedprocesses,itslinearnaturehasbeenitslargestproblem.Theprocessdoesnotdefinehow to respond tounexpectedoutput fromanyof the intermediateprocess.
Planning
Analysis Design Development Implement
ãJeffSutherland1993-2021 60
BarryBoehmintroducedaSpiralmethodologytoaddressthisproblem[51].Eachofthewaterfallphasesisendedwithariskassessmentandprototypingactivity.TheSpiralmethodologyisshowninFigure2.
The Spiral methodology “peels the onion,” progressing through “layers” of the development process. Aprototype letsusersdetermine if theproject isontrack,shouldbesentbacktopriorphases,orshouldbeended.However,thephasesandphaseprocessesarestilllinear.Requirementsworkisstillperformedintherequirementsphase,designworkinthedesignphase,andsoforth,witheachofthephasesconsistingoflinear,explicitlydefinedprocesses.
Figure2:SpiralMethodology
TheIterativemethodologyimprovesontheSpiralmethodology.EachiterationconsistsofallofthestandardWaterfall phases, but each iteration only addresses one set of parsed functionality. The overall projectdeliverablehasbeenpartitionedintoprioritizedsubsystems,eachwithcleaninterfaces.Usingthisapproach,onecantestthefeasibilityofasubsystemandtechnologyintheinitialiterations.Furtheriterationscanaddresourcestotheprojectwhilerampingupthespeedofdelivery.Thisapproachimprovescostcontrol,ensuresdeliveryofsystems(albeitsubsystems),andimprovesoverallflexibility.However,theIterativeapproachstillexpectsthattheunderlyingdevelopmentprocessesaredefinedandlinear.SeeFigure3.
Prototype
Evaluate alternatives;Identify, resolve risks
Determine objectives,alternatives,constraints
Plan next phases
Concept
Requirements
DesignImplement
Develop next level product
ãJeffSutherland1993-2021 61
Figure3:IterativeMethodology
Giventhecomplexenvironmentandtheincreasedrelianceonnew"state-of-the-art"systems,theriskenduredby system development projects has increased and the search for mechanisms to handle this risk hasintensified.
Onecanarguethatcurrentmethodologiesarebetterthannothing.Eachimprovesontheother.TheSpiralandIterativeapproaches implant formalriskcontrolmechanismsfordealingwithunpredictableresults.Aframeworkfordevelopmentisprovided.
However,eachrestsonthefallacythatthedevelopmentprocessesaredefined,predictableprocesses. Butunpredictableresultsoccurthroughouttheprojects.Therigorimpliedinthedevelopmentprocessesstiflestheflexibilityneededtocopewiththeunpredictableresultsandrespondtoacomplexenvironment.
System Test
Module Test
CodingDetail Design
PreliminaryDesign
RequirementsAnalysis
ãJeffSutherland1993-2021 62
Despitetheirwidespreadpresenceinthedevelopmentcommunity,ourexperienceintheindustryshowsthatpeopledonotusethemethodologiesexceptasamacroprocessmap,orfortheirdetailedmethoddescriptions.
Thefollowinggraphdemonstratesthecurrentdevelopmentenvironment,usinganyoftheWaterfall,SpiralorIterativeprocesses.Asthecomplexityofthevariablesincreaseeventoamoderatelevel,theprobabilityofa“successful” project quickly diminishes (a successful project is defined as a system that is useful whendelivered).SeeFigure4.
Figure4:DefinedProcessRisk/ComplexityGraph
4.SCRUMMETHODOLOGY
Thesystemdevelopmentprocessiscomplicatedandcomplex.Thereforemaximumflexibilityandappropriatecontrolisrequired.Evolutionfavorsthosethatoperatewithmaximumexposuretoenvironmentalchangeandhavemaximized flexibility. Evolution deselects thosewho have insulated themselves from environmentalchangeandhaveminimizedchaosandcomplexityintheirenvironment.
0.9
0.1
probability(Success) 0.5
Low Medium High
Complexity
Inflexible response to unpredictability (internal & external)causes sharp drop in p(Success)as complexity increases
ãJeffSutherland1993-2021 63
Anapproachisneededthatenablesdevelopmentteamstooperateadaptivelywithinacomplexenvironmentusing imprecise processes. Complex system development occurs under rapidly changing circumstances.Producing orderly systems under chaotic circumstances requires maximum flexibility. The closer thedevelopmentteamoperatestotheedgeofchaos,whilestillmaintainingorder,themorecompetitiveandusefultheresultingsystemwillbe.Langtonhasmodeledthiseffectincomputersimulations[6]andhisworkhasprovidedthisasafundamentaltheoremincomplexitytheory.
Methodologymaywellbethemostimportantfactorindeterminingtheprobabilityofsuccess.Methodologiesthatencourageandsupportflexibilityhaveahighdegreeoftoleranceforchangesinothervariables.Withthesemethodologies,thedevelopmentprocessisregardedasunpredictableattheonset,andcontrolmechanismsareputinplacetomanagetheunpredictability.
Ifwe graph the relationship between environmental complexity andprobability of successwith a flexiblemethodologythatincorporatescontrolsandriskmanagement,thetoleranceforchangeismoredurable.SeeFigure5.
Figure5:Risk/ComplexityComparisonGraph
ãJeffSutherland1993-2021 64
Figures4and5reflectsoftwaredevelopmentexperiencesatADM,Easel,VMARK,Borlandandvirtuallyeveryother developer of “packaged” software. These organizations have embraced risk and environmentalcomplexity during development projects. Increased product impact, successful projects, and productivitygainswereexperienced.Thebestpossiblesoftwareisbuilt.
WaterfallandSpiralmethodologiessetthecontextanddeliverabledefinitionatthestartofaproject.ScrumandIterativemethodologiesinitiallyplanthecontextandbroaddeliverabledefinition,andthenevolvethedeliverable during the project based on the environment. Scrum acknowledges that the underlyingdevelopmentprocessesareincompletelydefinedandusescontrolmechanismstoimproveflexibility.
Theprimarydifferencebetweenthedefined(waterfall,spiralanditerative)andempirical(Scrum)approachisthattheScrumapproachassumesthattheanalysis,design,anddevelopmentprocessesintheSprintphaseareunpredictable.Acontrolmechanismisusedtomanagetheunpredictabilityandcontroltherisk.Flexibility,responsiveness,andreliabilityaretheresults.SeeFigure6.
Figure6:ScrumMethodology
CharacteristicsofScrummethodologyare:
ãJeffSutherland1993-2021 65
• The firstand lastphases(PlanningandClosure)consistofdefinedprocesses,whereallprocesses,inputsandoutputsarewelldefined.Theknowledgeofhowtodotheseprocessesisexplicit.Theflowis linear, with some iterations in the planning phase.
• TheSprintphaseisanempiricalprocess.Manyoftheprocessesinthesprintphaseareunidentifiedoruncontrolled. It istreatedasablackboxthatrequiresexternalcontrols. Accordingly,controls,including risk management, are put on each iteration of the Sprint phase to avoid chaos whilemaximizing flexibility.
• Sprintsarenonlinearand flexible. Whereavailable,explicitprocessknowledge isused;otherwisetacitknowledgeandtrialanderrorisusedtobuildprocessknowledge.Sprintsareusedtoevolvethefinal product.
• TheprojectisopentotheenvironmentuntiltheClosurephase.ThedeliverablecanbechangedatanytimeduringthePlanningandSprintphasesoftheproject.Theprojectremainsopentoenvironmentalcomplexity, including competitive, time, quality, and financial pressures, throughout thesephases.
• Thedeliverableisdeterminedduringtheprojectbasedontheenvironment.
ThetableinFigure7comparestheprimaryScrumcharacteristicstothoseofothermethodologies.
Waterfall Spiral Iterative Scrum
Definedprocesses Required Required Required Planning & Closureonly
Finalproduct Determinedduringplanning
Determinedduringplanning
Set duringproject
Setduringproject
Projectcost Determinedduringplanning
Partiallyvariable Set duringproject
Setduringproject
Completiondate Determinedduringplanning
Partiallyvariable Set duringproject
Setduringproject
Responsiveness toenvironment
Planningonly Planningprimarily At endof eachiteration
Throughout
ãJeffSutherland1993-2021 66
Team flexibility,creativity
Limited-cookbookapproach
Limited-cookbookapproach
Limited -cookbookapproach
Unlimited duringiterations
Knowledgetransfer
Training prior toproject
Training prior toproject
Training priortoproject
Teamwork duringproject
Probability ofsuccess
Low Mediumlow Medium High
Figure7:MethodologyComparison
4.1SCRUMPHASES
Scrumhasthefollowinggroupsofphases:
4.1.1.PREGAME
• Planning:Definitionofanewreleasebasedoncurrentlyknownbacklog,alongwithanestimateofitsscheduleandcost.Ifanewsystemisbeingdeveloped,thisphaseconsistsofbothconceptualizationandanalysis.Ifanexistingsystemisbeingenhanced,thisphaseconsistsoflimitedanalysis.
• Architecture : Design how the backlog items will be implemented. This phase includes systemarchitecturemodificationandhighleveldesign.
4.1.2.GAME
• Development Sprints : Development of new release functionality, with constant respect to thevariablesoftime,requirements,quality,cost,andcompetition.Interactionwiththesevariablesdefinestheendofthisphase.Therearemultiple, iterativedevelopmentsprints,orcycles, thatareusedtoevolvethesystem.
4.1.3.POSTGAME
• Closure : Preparation for release, including final documentation, pre-release staged testing, andrelease.
ãJeffSutherland1993-2021 67
ãJeffSutherland1993-2021 68
ãJeffSutherland1993-2021 69
Figure8:ScrumMethodology
4.2PHASESTEPS
Eachofthephaseshasthefollowingsteps:
4.2.1.PLANNING
• Developmentofacomprehensivebackloglist.
• Definitionofthedeliverydateandfunctionalityofoneormorereleases.
• Selectionofthereleasemostappropriateforimmediatedevelopment.
• Mappingofproductpackets(objects)forbacklogitemsintheselectedrelease.
• Definitionofprojectteam(s)forthebuildingofthenewrelease.
• Assessmentofriskandappropriateriskcontrols.
• Reviewandpossibleadjustmentofbacklogitemsandpackets.
• Validationorreselectionofdevelopmenttoolsandinfrastructure.
• Estimationofreleasecost,includingdevelopment,collateralmaterial,marketing,training,androllout.
• Verificationofmanagementapprovalandfunding.
4.2.2.ARCHITECTURE/HIGHLEVELDESIGN
• Reviewassignedbacklogitems.
• Identifychangesnecessarytoimplementbacklogitems.
• Performdomainanalysistotheextentrequiredtobuild,enhance,orupdatethedomainmodelstoreflectthenewsystemcontextandrequirements.
• Refinethesystemarchitecturetosupportthenewcontextandrequirements.
• Identifyanyproblemsorissuesindevelopingorimplementingthechanges
ãJeffSutherland1993-2021 70
• Designreviewmeeting,eachteampresentingapproachandchangestoimplementeachbacklogitem.Reassignchangesasrequired.
4.2.3.DEVELOPMENT(SPRINT)
TheDevelopmentphaseisaniterativecycleofdevelopmentwork.Themanagementdeterminesthattime,competition,quality,or functionalityaremet, iterationsarecompletedandtheclosurephaseoccurs. ThisapproachisalsoknownasConcurrentEngineering.Developmentconsistsofthefollowingmacroprocesses:
• Meetingwithteamstoreviewreleaseplans.
• Distribution,reviewandadjustmentofthestandardswithwhichtheproductwillconform.
• IterativeSprints,untiltheproductisdeemedreadyfordistribution.
ASprintisasetofdevelopmentactivitiesconductedoverapre-definedperiod,usuallyonetofourweeks.Theintervalisbasedonproductcomplexity,riskassessment,anddegreeofoversightdesired.SprintspeedandintensityaredrivenbytheselecteddurationoftheSprint.Riskisassessedcontinuouslyandadequateriskcontrolsandresponsesareputinplace.EachSprintconsistsofoneormoreteamsperformingthefollowing:
• Develop: Defining changes needed for the implementation of backlog requirements into packets,openingthepackets,performingdomainanalysis,designing,developing,implementing,testing,anddocumentingthechanges. Developmentconsistsof themicroprocessofdiscovery, invention,andimplementation.
• Wrap:Closingthepackets,creatingaexecutableversionofchangesandhowtheyimplementbacklogrequirements.
• Review:All teamsmeeting topresentworkand reviewprogress, raisingand resolving issuesandproblems,addingnewbacklogitems.Riskisreviewedandappropriateresponsesdefined.
• Adjust: Consolidating the information gathered from the review meeting into affected packets,includingdifferentlookandfeelandnewproperties.
EachSprintisfollowedbyareview,whosecharacteristicsare:
• Thewholeteamandproductmanagementarepresentandparticipate.
ãJeffSutherland1993-2021 71
• Thereviewcanincludecustomers,sales,marketingandothers.
• Reviewcoversfunctional,executablesystemsthatencompasstheobjectsassignedtothatteamandincludethechangesmadetoimplementthebacklogitems.
• Thewaybacklogitemsareimplementedbychangesmaybechangedbasedonthereview.
• Newbacklog itemsmay be introduced and assigned to teams as part of the review, changing thecontentanddirectionofdeliverables.
• Thetimeofthenextreviewisdeterminedbasedonprogressandcomplexity.TheSprintsusuallyhaveadurationof1to4weeks.
4.2.4.CLOSURE
When themanagement team feels that the variables of time, competition, requirements, cost, and qualityconcurforanewreleasetooccur,theydeclaretherelease“closed”andenterthisphase.Theclosurephaseprepares thedevelopedproduct forgeneral release. Integration, system test,userdocumentation, trainingmaterialpreparation,andmarketingmaterialpreparationareamongclosuretasks.
4.3.SCRUMCONTROLS
Operatingattheedgeofchaos(unpredictabilityandcomplexity)requiresmanagementcontrolstoavoidfallingintochaos.TheScrummethodologyembodiesthesegeneral,loosecontrols,usingOOtechniquesfortheactualconstructionofdeliverables.
Riskistheprimarycontrol.Riskassessmentleadstochangesinothercontrolsandresponsesbytheteam.
ControlsintheScrummethodologyare:
• Backlog:Productfunctionalityrequirementsthatarenotadequatelyaddressedbythecurrentproductrelease. Bugs, defects, customer requested enhancements, competitive product functionality,competitive edge functionality, and technology upgrades are backlog items.
• Release/Enhancement:backlogitemsthatatapointintimerepresentaviablereleasebasedonthevariablesofrequirements,time,quality,andcompetition.
ãJeffSutherland1993-2021 72
• Packets:Productcomponentsorobjectsthatmustbechangedtoimplementabacklogitemintoanewrelease.
• Changes: Changes that must occur to a packet to implement a backlog item.
• Problems: Technical problems that occur and must be solved to implement a change.
• Risks:risksthataffectthesuccessoftheprojectarecontinuouslyassessedandresponsesplanned.Other controls are affected as a result of risk assessment.
• Solutions: solutions to the problems and risks, often resulting in changes.
• Issues: Overall project and project issues that are not defined in terms of packets, changes andproblems.
ThesecontrolsareusedinthevariousphasesofScrum.Managementusesthesecontrolstomanagebacklog.Teamsusethesecontrolstomanagechanges,problems.Bothmanagementandteamsjointlymanageissues,risks,andsolutions.Thesecontrolsarereviewed,modified,andreconciledateverySprintreviewmeeting.
4.4SCRUMDELIVERABLES
The delivered product is flexible. Its content is determined by environment variables, including time,competition,cost,or functionality.Thedeliverabledeterminantsaremarket intelligence,customercontact,andtheskillofdevelopers.Frequentadjustmentstodeliverablecontentoccurduringtheprojectinresponsetoenvironment.Thedeliverablecanbedeterminedanytimeduringtheproject.
4.5SCRUMPROJECTTEAM
Theteamthatworksonthenewreleaseincludesfulltimedevelopersandexternalpartieswhowillbeaffectedby thenewrelease, suchasmarketing, sales, and customers. In traditional releaseprocesses, these lattergroups are kept away from development teams for fear of over-complicating the process and providing“unnecessary” interference. The Scrum approach, however, welcomes and facilitates their controlledinvolvement at set intervals, as this increases the probability that release content and timing will beappropriate,useful,andmarketable.
ãJeffSutherland1993-2021 73
Thefollowingteamsareformedforeachnewrelease:
Management:LedbytheProductManager,itdefinesinitialcontentandtimingoftherelease,thenmanagestheir evolution as the project progresses and variables change.Management dealswith backlog, risk, andreleasecontent.
Developmentteams:Developmentteamsaresmall,witheachcontainingdevelopers,documentersandqualitycontrolstaff. Oneormoreteamsofbetweenthreeandsixpeopleeachareused. Eachisassignedasetofpackets(orobjects),includingallbacklogitemsrelatedtoeachpacket.Theteamdefineschangesrequiredtoimplementthebacklogiteminthepackets,andmanagesallproblemsregardingthechanges.Teamscanbeeither functionally derived (assigned those packets that address specific sets of product functionality) orsystemderived(assigneduniquelayersofthesystem).Themembersofeachteamareselectedbasedontheirknowledgeandexpertiseregardingsetsofpackets,ordomainexpertise.
4.6SCRUMCHARACTERISTICS
TheScrummethodologyisametaphorforthegameofRugby.RugbyevolvedfromEnglishfootball(soccer)undertheintensepressureofthegame:
RugbystudentWilliamWebbEllis,17,inauguratesanewgamewhoseruleswillbecodifiedin1839.Playingsoccerforthe256-year-oldcollegeinEastWarwickshire,Ellisseesthattheclockisrunningoutwithhisteambehindsohescoopsuptheballandrunswithitindefianceoftherules.
ThePeople'sChronology,HenryHoltandCompany,Inc.Copyright©1992.
Scrumprojectshavethefollowingcharacteristics:
• Flexible deliverable - the content of the deliverable is dictated by the environment.
• Flexible schedule - the deliverable may be required sooner or later than initially planned.
• Smallteams-eachteamhasnomorethan6members.Theremaybemultipleteamswithinaproject.
ãJeffSutherland1993-2021 74
• Frequentreviews- teamprogress isreviewedas frequentlyasenvironmentalcomplexityandriskdictates(usually1to4weekcycles).Afunctionalexecutablemustbepreparedbyeachteamforeachreview.
• Collaboration - intra and inter-collaboration is expected during the project.
• ObjectOriented-eachteamwilladdressasetofrelatedobjects,withclearinterfacesandbehavior.
TheScrummethodologysharesmanycharacteristicswiththesportofRugby:
• The context is set by playing field (environment) and Rugby rules (controls).
• The primary cycle is moving the ball forward.
• Rugby evolved from breaking soccer rules - adapting to the environment.
Thegamedoesnotenduntilenvironmentdictates(businessneed,competition,functionality,timetable).
5.ADVANTAGESOFTHESCRUMMETHODOLOGY
Traditionaldevelopmentmethodologiesaredesignedonlytorespondtotheunpredictabilityoftheexternalanddevelopmentenvironmentsatthestartofanenhancementcycle.SuchnewerapproachesastheBoehmspiralmethodologyanditsvariantsarestilllimitedintheirabilitytorespondtochangingrequirementsoncetheprojecthasstarted.
TheScrummethodology,ontheotherhand, isdesignedtobequiteflexiblethroughout.Itprovidescontrolmechanisms for planning a product release and then managing variables as the project progresses. Thisenables organizations to change the project and deliverables at any point in time, delivering the mostappropriaterelease.
TheScrummethodologyfreesdeveloperstodevisethemostingenioussolutionsthroughouttheproject,aslearningoccursandtheenvironmentchanges.
ãJeffSutherland1993-2021 75
Small,collaborativeteamsofdevelopersareabletosharetacitknowledgeaboutdevelopmentprocesses.Anexcellenttrainingenvironmentforallpartiesisprovided.
ObjectOrientedtechnologyprovidesthebasisfortheScrummethodology.Objects,orproductfeatures,offera discrete and manageable environment. Procedural code, with its many and intertwined interfaces, isinappropriatefortheScrummethodology.Scrummaybeselectivelyappliedtoproceduralsystemswithcleaninterfacesandstrongdataorientation.
6.SCRUMPROJECTESTIMATING
Scrumprojectscanbeestimatedusingstandardfunctionpointestimating.However,itisadvisabletoestimateproductivityatapproximatelytwicethecurrentmetric.Theestimateisonlyforstartingpurposes,however,sincetheoveralltimetableandcostaredetermineddynamicallyinresponsetotheenvironmentalfactors.
OurobservationshaveledustoconcludethatScrumprojectshavebothvelocityandacceleration.Intermsoffunctionsdelivered,orbacklogitemscompleted:
• initialvelocityandaccelerationarelowasinfrastructureisbuilt/modified
• asbasefunctionalityisputintoobjects,accelerationincreases
• accelerationdecreasesandvelocityremainssustainablyhigh
Furtherdevelopmentinmetricsforempiricalprocessesisrequired.
7.SYSTEMDEVELOPMENTMETHODOLOGIES:DEFINEDOREMPIRICAL
System development is the act of creating a logical construct that is implemented as logic and data oncomputers.Thelogicalconstructconsistsofinputs,processes,andoutputs,bothmacro(wholeconstruct)andmicro(intermediatestepswithinwholeconstruct).Thewholeisknownasanimplementedsystem.
ãJeffSutherland1993-2021 76
Many artifacts are created while building the system. Artifacts may be used to guide thinking, checkcompleteness,andcreateanaudittrail.Theartifactsconsistofdocuments,models,programs,testcases,andotherdeliverablescreatedpriortocreatingtheimplementedsystem.Whenavailable,ametamodeldefinesthesemanticcontentofmodelartifacts.Notationdescribesthegraphinganddocumentationconventionsthatareusedtobuildthemodels.
Theapproachusedtodevelopasystemisknownasamethod.Amethoddescribestheactivitiesinvolvedindefining,building,andimplementingasystem;amethodisaframework.Sinceamethodisalogicalprocessforconstructingsystems(process),itisknownasametaprocess(aprocessformodelingprocesses).
Amethod has micro andmacro components. Themacro components define the overall flow and time-sequencedframeworkforperformingwork.Themicrocomponentsincludegeneraldesignrules,patternsandrulesofthumb.Generaldesignrulesstatepropertiestoachieveortoavoidinthedesignorgeneralapproachestotakewhilebuildingasystem.Patternsaresolutionsthatcanbeappliedtoatypeofdevelopmentactivity;theyaresolutionswaitingforproblemsthatoccurduringanactivityinamethod.Rulesofthumbconsistofageneralbodyofhintsandtips.
Figure9:RelationshipbetweentheMethod,theArtifacts,andtheSystem
ãJeffSutherland1993-2021 77
Applying concepts from industrial process control to the field of systems development, methods can becategorizedaseither“theoretical”(fullydefined)or“empirical”(blackbox).
Correctly categorizing systems developmentmethods is critical. The appropriate structure of amethod forbuildingaparticulartypeofsystemdependsonwhetherthemethodistheoreticalorempirical.
Models of theoretical processes are derived from first principles, using material and energy balances andfundamentallawstodeterminethemodel.Forasystemsdevelopmentmethodtobecategorizedastheoretical,itmustconformtothisdefinition.
Modelsofempiricalprocessesarederivedcategorizingobservedinputsandoutputs,anddefiningcontrolsthatcausethemtooccurwithinprescribedbounds.Empiricalprocessmodelinginvolvesconstructingaprocessmodelstrictlyfromexperimentallyobtainedinput/outputdata,withnorecoursetoanylawsconcerningthefundamental nature and properties of the system. Noa priori knowledge about the process is necessary(althoughitcanbehelpful);asystemistreatedlikeablackbox.
PrimarycharacteristicsofboththeoreticalandempiricalmodelingaredetailedinFigure10.
TheoreticalModeling EmpiricalModeling
1.Usuallyinvolvesfewermeasurements;requiresexperimentation only for the estimation ofunknownmodelparameters.
Requires extensive measurements, because itreliesentirelyonexperimentationforthemodeldevelopment.
2.Providesinformationabouttheinternalstateoftheprocess.
Provides informationonly about thatportionofthe processwhich can be influenced by controlaction.
3. Promotes fundamental understanding of theinternalworkingsoftheprocess.
Treatstheprocesslikea“blackbox.”
4.Requiresfairlyaccurateandcompleteprocessknowledge.
Requires no such detailed knowledge; only thatoutput data be obtainable in response to inputchanges.
5.Not particularly useful for poorly understoodand/orcomplexprocesses.
Quiteoftenprovestobetheonlyalternative formodeling the behavior of poorly understoodand/orcomplexprocesses.
ãJeffSutherland1993-2021 78
6.Naturallyproducesboth linear andnonlinearprocessmodels.
Requires special methods to produce nonlinearmodels.
Figure10:Theoreticalvs.empiricalmodeling
Uponinspection,weassertthatthesystemsdevelopmentprocessisempirical:
• Applicablefirstprinciplesarenotpresent
• Theprocessisonlybeginningtobeunderstood
• Theprocessiscomplex
• Theprocessischanging
Mostmethodologistsagreewiththisassertion;“...youcan’texpectamethodtotellyoueverythingtodo.Writingsoftwareisacreativeprocess,likepaintingorwritingorarchitecture......(amethod)suppliesaframeworkthattellshowtogoaboutitandidentifiestheplaceswherecreativityisneeded.Butyoustillhavetosupplythecreativity....”[52]
Categorizing the systemsdevelopmentmethodsas empirical is critical to theeffectivemanagementof thesystemsdevelopmentprocess.
If systems development methods are categorized as empirical, measurements and controls are requiredbecause it isunderstoodthat the innerworkingsof themethodareso looselydefinedthat theycannotbecountedontooperatepredictably.
Inthepast, methodshavebeenprovidedandappliedasthoughtheyweretheoretical. Asaconsequence,measurementswerenotrelieduponandcontrolsdependentuponthemeasurementsweren’tused.
Manyoftheproblemsindevelopingsystemshaveoccurredbecauseofthisincorrectcategorization.Whenablackboxprocessistreatedasafullydefinedprocess,unpredictableresultsoccur.Also,thecontrolsarenotinplacetomeasureandrespondtotheunpredictability.
ãJeffSutherland1993-2021 79
CHAPTER4:GETTINGSTARTEDWITHSCRUM
Scrumhasthreeroles,threeceremonies,andthreeartifacts:
• Roles–ProductOwner,ScrumMaster,Team
• Ceremonies–SprintPlanning,DailyScrum,SprintReview
• Artifacts–ProductBacklog,SprintBacklog,BurndownChart
Inorder togetstartedwithScrum,everyoneneeds tounderstandrolesandresponsibilitiesaswellas theoperationalflowofaScrum.UnitsofworkarebrokendownintodiscreteintervalscallSprints(sometimescallediterations)of30daysorless.Practicalexperiencehasshownthatteamscannoteffectivelymaintainaglobalviewofallthepiecesofaprojectinmorethan30daychunks.AllAgileprocessesuseiterationsthatdeliverworkingsoftware that is “done”at theendofan iteration in thesense that featurescompletedarepotentiallyshippabletoendusers.Thismeansthesoftwaremustbetestedtoensurethatitworks.Today,onlyabouthalftheScrumteamsworldwidecanproducetestedsoftwareattheendofaSprint.ExtensivedatafromaCMMILevel5companyshowsthatthisfailuremodetendstodoubledefectsanddoubledevelopmenttime.
ToeffectivelydeliverpotentiallyshippableproductattheendoftheSprint,experiencehasshowthat:
1. TheremustbeoneProductOwnerthatdeliverstheteamaclearlistofconsistentprioritiesthatareheldfixedforthelengthofaSprint.Decadesofexperienceinsoftwareengineeringhaveshownthatgiving a development team multiple priorities or changing priorities during an iteration causesdelayeddecisionsandexcessiverework.Morethan50%reductioninproductivityofsuchteamsisthenorm. Many development teams have doubled their productivity in the first month of Scrumimplementationbyfixingthisproblem.
2. TheProductOwnermusthaveaProductBacklogoffeaturesthatarerankorderedbybusinessvalue.
3. Theworkrequiredtoimplementthefeaturesmustbeestimatedbytheteamthatwillimplementthefeatures.Researchhasshownthatwhileexpertsmayaccuratelyestimatehowlongitwouldtakethemtoimplementafeature,itisimpossibleforthemtoestimatethetechnicalanddomainknowledgeofthedelivery team, thecompetingpriorities fromotherprojects that impact teamproductivity, thehumanfactorsonateamsuchasmotivation,conflictresolution,teamspiritthatvaryovertime,orthelossoradditionofpersonneltoateambecauseofhiring,firing,vacations,orotherfactorsthatdisruptteamstability.
4. TheteammustmaintainaBurndownChartandknowhowmanyfeaturescanbedeliveredattheendofeachSprint.TheestimateofthenumberoffeaturesthatcanbedeliveredattheendofaSprintisknowastheteam’s“velocity.”SprintsneedtobethesamefixedlengthsotheProductOwnercanusethevelocitytobuildareleaseplan.Managementandcustomersneedtoknowthedateofshipmentofanewrelease.WithoutfixedlengthSprintswithknownvelocity,releasedatesarenotpredictable.
ãJeffSutherland1993-2021 80
5. TheteammustbeabletoexecutetheSprintwithoutbeingdisruptedbyprojectleaders,management,orotherchaosinducingforcesinanorganization.
Whileonlyabout10%ofScrumteamswordwidecanmeetthesebasicstandardstoday,anydescriptionofhowtogetstartedwithScrummustaddresshowtostepbystepmovetowardsthesebasicScrumpractices.
ãJeffSutherland1993-2021 81
SCRUMONLARGEPROJECTS:DISTRIBUTED,OUTSOURCEDSCRUM
FIRSTSCRUMSCALINGATIDXSYSTEMS1996-2000In1996,JeffSutherlandbecomeSVPofProductDevelopmentatIDXSystemsCorporationwithadevelopmentteamthatgrewtoalmost600peopleby2000.Scrumwasintroducedimmediatelyforalldevelopers.Aneworganizational structure was created to coordinate Scrum teams called a Scrum of Scrums. Previousmanagement positions were eliminated and all managers became team leaders. Typically, Directors ofEngineeringbecameScrumofScrumleaders.KenSchwaberprovidedconsultingexperienceinvariouspartsofthedevelopmentorganizationduring1996-2000tohelpintroduceScrumtotheorganization.
Businessunitswithproductportfoliostypicallyhadlessthan100developers.AScrumofScrumswasabletomanagethissizegroupeffectively.AtthislevelaleadarchitectbecametheProductOwnerofthearchitecturebacklogforthebusinessunitandwasthesinglepersonresponsibleforthearchitectureforthatunit.
AtaSVP level, therewasa teamofDirectorsandVPs thatmetperiodically to coordinateactivitiesacrossbusinessunits.TheSVP,havingssystemsarchitecturebackground,ledavirtualteamofallthebusinessunitarchitectstodevelopaglobalarchitecturebacklogforallbusinessunits.SprintplanninginvolvedtheentirebusinessunitonceamonthandthebusinessunitarchitectwasresponsibleforgettingacommitmentfromeachScrumteamtodevote10%oftheresourcesineveryScrumteamtoaddressingtheglobalarchitecturebacklog.Thisdroveallbusinessunitsincrementallytowardsacommonarchitecturalframework.
Asimilarvirtualteamstrategywasusedforsoftwareintegrationacrossbusinessunitsandforcommonqualityassuranceprocessesandprocedures.ThevirtualteamstrategyallowedallseniordeveloperstobeworkingwithaScrumteamduringeverySprint.Theapproachwasdesignedtoavoidseparatespecializedteamsandtogeteveryoneintofrontlineproduction.
TheSVPteamworkedwelltocoordinateScrumofScrumsteamsacrossbusinessunits.However,itwasnotoptimalfordrivingaglobalProductBacklogacrossbusinessunits.Today,bestpracticesaretoimplementaMetaScrum above the Scrum of Scrums teams. The MetaScrum is lead by the Chief Product Owner andincorporatesallstakeholdersinthecompany.ProductreleasestrategyandthestateofeverySprintisreviewedatMetaScrummeetings.Alldecisionstostart,stop,orchangeSprintsaremadethere.Often,theCEOistheScrumMasterfortheMetaScrum.
LINEARSCALABILITYINLARGE,DISTRIBUTED,ANDOUTSOURCEDSCRUMSTwocasestudiespublishedin2007demonstrateforthefirsttimethatasoftwaredevelopmentprocesscanscalelinearlyacrossbothdevelopmentteamsizeandgeographies.TheSirsiDynix/StarSoftDevelopmentLabsprojectdeliveredoveramillionlinesofcodewithteamsdistributedacrosstheU.S.,Canada,andRussia[19].
ãJeffSutherland1993-2021 82
WhenthedevelopmentteamdoubledinsizebybringingonengineersinSt.Petersburg,Russia,thevelocityofsoftwaredeliverymorethandoubled.
AsimilareffectwasnotedbyaCMMILevel5ScrumimplementationatSystematicSoftwareEngineeringinDenmark [20]. IntroducingScrum increasedproductivity is small teamsonlyslightlyas theywerealreadyAgile.Onlargerprojects,theyconsistentlyachievedthesamelevelofproductivityperdeveloperasonsmallteams.Afterachievingat80%reductioninplanningcosts,a50%reductionintotalprojectcosts,alongwithsignificantly increased user and employee satisfaction, they converted their company-wide CMMI Level 5processdocumentation,training,andimplementationtoScrum.
AtAgile2008,SystematicSoftwareEngineeringpublishedamoredetailedpaperoncombiningCMMIwithScrumtoproduceamoredisciplinedenvironmentforlargeprojectswhileachievingthehighvelocityseenontheirsmallScrumprojects.ThispaperisincludedandshowshowSystematichasinstitutionalizedacompanywideCMMILevel5Scrumprocess.
TherearemanymovingpartsinanenterprisewideimplementationofScrum.Whileaclear,consistentmodelisachievableforanycompany,itmustbelocalizedintoaspecificcompanystructurethroughinspectionandadaptation,thehallmarkofScrum.ThisimmediatelyleadstoquestionsonhowtobestorganizethecompanytotakeadvantageofScrum.ItisstronglyrecommendedtouseexpertisefromanexperiencedtrainerwhohasledmultipleenterpriseScrumimplementationstoworkthroughbeststrategiesforimplementingenterpriseScruminaspecificcompany.
ãJeffSutherland1993-2021 83
AGILECANSCALE:INVENTINGANDREINVENTINGSCRUMINFIVECOMPANIES
JeffSutherland,Ph.D.
PatientKeeper,Inc.,2001
INTRODUCTIONAgiledevelopmentisfocusedondeliveringmaximumbusinessvalueintheshortestpossibletime.Itiswellknownthatoverhalftherequirementsonatypicalsoftwareprojectchangeduringdevelopmentandabouthalfthefeaturesindeliveredsoftwareareneverusedbycustomers.Adaptiveplanningandself-organizingteamsareusedtoembracechangingrequirementsandavoidbuildingfeatureswithlowreturnoninvestment.Theresultisfasterdelivery,betterquality,higherusersatisfaction,andamorecreativeandenjoyableworkingenvironmentfordevelopers.
ScrumderivesfromJapanesebestpracticesinleanmanufacturingandleanproductdevelopment[1,14].ThegoalofScrumis toachieve the“ToyotaEffect”,deliver four timesasmuchsoftwarewith twelve times thequalitywithinaseriesofshorttimeboxescalled“Sprint's,”whichlast30daysorless.Scrumischaracterizedby15minute,intensive,dailymeetingsofamultidisciplinarysoftwaredeliveryteam,usuallyincludingproductmarketing,testers,softwareanalysts,designers,andcoders,andevendeploymentsupportstaff.PriortoeachiterationalistoffeaturestobedeliveredcalledtheProductBacklogisreprioritizedsothatmaximumvaluefeatures are developed first. Most tasks required to complete Product Backlog features are defined andestimatedatthestartoftheSprintbythemultidisciplinarydevelopmentteamtoproducttheSprintBacklog.SometasksrequirefurtherdesignordefinitionduringaSprintsotheSprintBacklogwilldynamicallyevolve,yetaskilledteamwillestimatetheamountofworkrequiredtocompleteaSprintwithlessthan20%errorusingimproveestimatingtechniquesbasedonthelatestresearchonplanning[31].
Aleanproductionenvironmentisbasedona“pull”system[8]whereteammemberspullinventoryintoaworkenvironment“justintime.”AthestartofaSprinttheteampullstheamountofProductBacklogintotheSprintthattheycancommittocompleteduringtheSprint.LoadingofteammembersduringtheSprintisdynamic,inthesensethatteammemberschosetheirowntasksand“pull”fromtheSprintBacklogwhentheyarereadytostartanothertask.Goodteams“minimizetheworkinprogress”bykeepingasfewtasksopenaspossibletoavoidintegrationproblemsandmisseddatesattheendofaSprint.
WithinaSprint,problemsandchallengesareevaluatedinthedailyScrummeetingandtheteamselforganizesinrealtimetomaximizethenumberofproductionreadyfeaturesdeliveredbytheteamattheendofeachSprint.Thedailymeetingsfocusedonanswerstothreequestionsbyeachmemberoftheteam–WhatdidIdoyesterday?What will I do today?What blocks, problems, or impediments are getting inmyway? AwellfunctioningteamwilldynamicallyrecalculatetheplandailythroughabriefdiscussionandprovideenoughinformationforaScrumMaster,theteamleader,tocalculatea“BurndownChart”ofworktobecompleted.Teameffortstoaccelerateordeceleratethedownwardvelocityoftheburndowngraphallowateamto“fly”theprojectintoafixeddeliverydate.
ãJeffSutherland1993-2021 84
A typical Burndown Chart is illustrated below. It consists of the cumulative time it takes to completeoutstandingtasksfordeliverablesoftwareforaScrumsprint.Eachdeveloperbreaksdowntasksintosmallpiecesandentersintoabacklogsystemworkremainingoneachactivetaskeveryday.Thiscouldbeassimpleas task updates on a white board or an Excel spreadsheet, or could be a specialized tool for support ofdistributedteams.Theremainingworkforeachtaskisaddeduptogeneratethecumulativebacklog. Bestcurrentmethodsrequireonlyoneminuteofeachdeveloperstimeeachdaytoupdatetwodataitemsforactivetasks(percentcompleteandtimeremaining)allowinganautomatedsystemtoproducetheBurndownChart,showinghowfastoutstandingbacklogisdecreasingeachday.InthedailyScrummeetings,theteammembersuseinformationsharingtodeterminewhatactionstakenthatdaywillmaximizethedownloadmovementofthecumulativebacklog.ExperiencehasshownthatScrumprojectplanningwillconsistentlyproduceafasterpathtotheendgoalthananyotherformofprojectplanningreportedtodate,withlessadministrativeoverheadthananypreviouslyreportedapproach.
FIGURE10:BURNDOWNCHART
Details of the Scrum approach have been carefully documented elsewhere [23]. Scrum is the only Agilemethodologythathasbeenformalizedandpublishedasanorganizationalpatternforsoftwaredevelopment[26].Theprocessassumesthatrequirementswillchangeduringtheperiodbetweeninitialspecificationanddeliveryofaproduct.ItsupportsHumphrey’sRequirementsUncertaintyPrinciple[53],whichstatesthatforanewsoftwaresystem,therequirementswillnotbecompletelyknowuntilaftertheusershaveusedit.ScrumallowsforZiv’sUncertaintyPrincipleinsoftwareengineering,whichobservesthatuncertaintyisinherentandinevitableinsoftwaredevelopmentprocessesandproducts[54].AnditaccountsforWegner’smathematicalproof(lemma)thatitisnotpossibletocompletelyspecifyaninteractivesystem[55].Mostsoftwaresystems
01020304050
1 2 3 4 5 6 7 8 9 10
Burndown Chart
ãJeffSutherland1993-2021 85
built today are object-oriented implementations, and most of those object-oriented systems depend onenvironmentalinputstodetermineprocessoutputs(i.e.,theyareinteractivesystems).
Traditional,heavyweightsoftwaremethodologiesassumethatrequirementscanbespecifiedinadvance,thattheywillnotchangeduringdevelopment, that theusersknowwhat theywantbefore theysee it,andthatsoftwaredevelopmentisapredictable,repeatableprocess.Theseassumptionsarefundamentallyflawedandinconsistentwiththemathematicallemmasandprinciplescitedabove.Asaresult,31%ofsoftwareprojects,usuallydrivenbyavariantofthewaterfallmethodology,areterminatedbeforecompletion[56].
ThisarticleservesasashortretrospectiveontheoriginsofScrum,itsevolutioninfivecompanies,andafewkeylearningsalongtheway.ItwillprovideareferencepointforfurtherinvestigationandimplementationofScrum for those interested inusingaproven, scalable, lightweightdevelopmentprocess that supports theprinciples of the Agile Alliance as outlined in the “Manifesto for Agile Software Development” (seewww.agilealliance.org).
EASELCORPORATION:THEFIRSTSCRUMScrumwasstartedin1993forsoftwareteamsatEaselCorporation,whereIwasVPofobjecttechnology.Inthe initial Scrum, we built the first object-oriented design and analysis tool that incorporated round-tripengineering. The second Scrum implemented the first product to completely automate object-relationalmappinginenterprisedevelopmentenvironment.Iwasassistedbytwoworld-classdevelopers--JeffMcKenna,nowaScrumandeXtremeProgrammingconsultant,andJohn'sScumniotales,nowadevelopmentleaderforobject-orienteddesigntoolsatRationalCorporation.
In1995,EaselwasacquiredbyVMARK. Scrumcontinued thereuntil I joined Individual in1996asVPofengineeringtodevelopPersonalNewsPage(nowwww.office.com).IaskedKenSchwaber,CEOofAdvancedDevelopmentMethodologies, tohelpme incorporateScrumintoIndividual’sdevelopmentprocess. Onthesameyear,ItookScrumtoIDXSystemswhenIassumedthepositionofseniorVPofengineeringandproductdevelopmentandCTO.IDX,oneofthelargestUShealthcaresoftwarecompanies,wastheprovinggroundformultiple team Scrum implementations. At one point, almost 600 developerswereworking on dozens ofproducts. In2000,ScrumwasintroducedtoPatientKeeper,amobile/wirelesshealthcareplatformwhereIbecameCTO.SoIhaveexperiencedScruminfivecompaniesthatvariedwidelyinsize.TheywereprovinggroundsforScruminallphasesofcompanygrowthfromstartup,toinitialIPO,tomid-sized,andthentoalargecompanydeliveringenterprisesystemstothemarketplace.
“ALL-AT-ONCE”SOFTWAREDEVELOPMENTThereweresomekeyfactorsthatinfluencedtheintroductionofScrumatEaselCorporation.ThebookWickedProblems,RighteousSolutions[32]byPeterDeGraceandLeslieHuletStahlreviewedthereasonswhythewaterfallapproachtosoftwaredevelopmentdoesnotworkforsoftwaredevelopmenttoday.Requirementsarenotfullyunderstoodbeforetheprojectbegins.Theusersknowwhattheywantonlyaftertheyseeaninitial
ãJeffSutherland1993-2021 86
versionofthesoftware.Requirementschangeduringthesoftwareconstructionprocess.Andnewtoolsandtechnologiesmakeimplementationstrategiesunpredictable.DeGraceandStahlreviewed“All-at-Once”modelsofsoftwaredevelopment,whichuniquelyfitobject-orientedimplementationofsoftwareandhelptoresolvethesechallenges.
“All-at-Once”modelsofsoftwaredevelopmentassumethatthecreationofsoftwareisdonebysimultaneouslyworkingonrequirements,analysis,design,coding,andtestingandthendeliveringtheentiresystemallatonce.The simplestAll-at-Oncemodel is a single super-programmer creating anddelivering an application frombeginningtoend.Allaspectsofthedevelopmentprocessresideinasingleperson’shead.Thisisthefastestway to deliver a product that has good internal architectural consistency, and it is the “hacker”mode ofimplementation. The next level of approach to All-at-Once development is handcuffing two programmerstogether,asintheXPpracticeofpairprogramming[24].Twodevelopersdelivertheentiresystemtogether.Thisisbeenshowntodeliverbettercode(intermsofusability,maintainability,flexibility,andextendability)fasterthanworkdeliveredbylargerteams.Thechallengeistoachieveasimilarproductivityeffectinthelargewithanentireteamandthenwithteamsofteams.
OurteambasedAll-at-OncemodelwasbasedonboththeJapaneseapproachtonewproductdevelopment,Sashimi,andScrum.Wewerealreadyusingproductionprototypingtobuildsoftware.Itwasimplementedinslices(Sashimi)whereanentirepieceoffullyintegratedfunctionalityworkedattheendofaniteration.WhatintrigueduswasHirotakaTakeuchiandHujiroNonaka’sdescriptionoftheteam-buildingprocessinsettingupandmanagingaScrum[1].Theideaofbuildingaself-empoweredteaminwhicheveryonehadaglobalviewoftheproductonadailybasisseemedliketherightidea.Thisapproachtomanagingtheteam,whichhadbeensosuccessfulatHonda,Canon,andFujitsu,resonatedwiththesystemsthinkingapproachbeingpromotedbyPeterSengeatMIT[16].
Wewerealsoimpactedbyrecentpublicationsincomputerscience.AsIalludedabove,PeterWagneratBrownUniversitydemonstratedthatitwasimpossibletofullyspecifyortestaninteractivesystem,whichisdesignedtorespondtoexternal inputs(Wegner's lemma)[55]. Herewasmathematicalproof thatanyprocess thatassumedknowninputs,asdoesthewaterfallmethod,wasdoomedtofailurewhenbuildinganobject-orientedsystem.
WewereproddedintosettingupthefirstScrummeetingafterreadingJamesCoplien'spaperonBorland'sdevelopmentofQuattroProforWindows[2].TheQuattroteamdeliveredonemillionlinesofC++codein31months,withafourpersonstaffgrowingtoeightpeoplelaterintheproject.Thiswasaboutathousandlinesofdeliverablecodeperpersonperweek,probablythemostproductiveprojecteverdocumented.Theteamattainedthislevelofproductivitybyintensiveinteractionindailymeetingswithprojectmanagement,productmanagement,developers,documenters,andqualityassurancestaff.
SOFTWAREEVOLUTIONANDPUNCTUATEDEQUILIBRIUM
ãJeffSutherland1993-2021 87
OurdailymeetingsatEaselweredisciplinedinawaythatwenowunderstandastheScrumpattern[26].Themost interesting effect of Scrum on Easel's development environment was an observed "punctuatedequilibriumeffect."Afullyintegratedcomponentdesignenvironmentleadstorapidevolutionofasoftwaresystemwithemergent,adaptiveproperties, resembling theprocessofpunctuatedequilibriumobserved inbiologicalspecies.
Evolutionary biologists have noticed that change occurs sharply at intervals separated by long periods ofapparent stagnation, leading to the concept of punctuated equilibrium [6]. Computer simulations of thisphenomenon suggest that periods of equilibrium are actually periods of ongoing genetic change of anorganism.Theeffectsofthatchangearenotapparentuntilseveralsubsystemsevolveinparalleltothepointwheretheycanworktogethertoproduceadramaticexternaleffect[10].Thispunctuatedequilibriumeffecthasbeenobservedby teamsworking in a component-basedenvironmentwithadequatebusinessprocessengineeringtools,andtheScrumdevelopmentprocessaccentuatestheeffect.
Byhavingeverymemberoftheteamseeeverydaywhateveryotherteammemberwasdoing,webegantoseehowwecouldaccelerateeachother'swork.Forinstance,onedevelopercommentedthatifhechangedafewlines incode,hecouldeliminatedaysofwork foranotherdeveloper. Thiseffectwassodramatic that theprojectacceleratedthepointwhereithadtobesloweddown.ThishyperproductivestatewasseeninseveralsubsequentScrum's,butneverwentsodramaticastheoneatEasel.
STIMULATINGSOFTWAREEVOLUTIONWITHSYNCSTEPS
ThefirstScrumworkedfromauniqueviewofthesoftwaresystem.Aprojectdomaincanbeviewedasasetofpackagesthatwillformarelease.Packagesarewhattheuserperceivesaspiecesoffunctionality,andtheyevolveoutofworkontopicareas(seeFigure2).Topicareasarebusinessobjectcomponents.Changesareintroducedintothesystembyintroducingaunitofworkthataltersacomponent.Refactoringoftencausesasinglechangetoripplethroughoutthesystem.ThisunitofworkintheinitialScrumwascalledaSynchStep.
ãJeffSutherland1993-2021 88
Figure2:ViewofasoftwaresystemsasseenbythefirstScrum
SystemevolutionproceedsinSyncSteps(seeFigure3).AfteroneormoreSyncStepshavegonetocompletionandforcedsomerefactoringthroughoutthesystem,anewpackageoffunctionalityemergesthatisobservabletotheuser.TheseSyncStepsaresimilartogeneticmutations.Typically,severalinterrelatedcomponentsmustmutateinconcerttoproduceasignificantnewpieceoffunctionality.Thisnewfunctionalityappearsasthepunctuatedequilibriumeffecttobuildersinthesystem.Foraperiodoftime,thesystemisstablewithnonewbehavior.Thenwhenacertain(somewhatunpredictable)SyncStepcompletes,thewholesystempopsuptoanewleveloffunctionality,oftensurprisingthedevelopmentteam.
ãJeffSutherland1993-2021 89
Figure3:FiringaSyncStepcausesaripplethroughthesystemthattriggersemissionofapackageoffunctionalityvisibletotheuser.
This aspect of self-organization is nowunderstood as a type of Set-BasedConcurrent Engineering (SBCE)whichispracticedatToyota[57].Developersconsidersetsofpossiblesolutionsandgraduallynarrowthesetofpossibilitiestoconvergeonafinalsolution.Here,decisionsonhowandwheretoimplementafeatureinasetofcomponentswasdelayeduntilthelastpossiblemoment.Themostevolvedcomponentisselected“justin time” to absorb new functionality, resulting in minimal coding and a more elegant architecture. Thusemergentarchitecture,acoreprincipleinallAgileprocesses,isnotrandomevolution.Properlyimplemented,itisanSBCEtechniqueviewedasabestbusinesspracticeinsomeoftheworld’sleadingcorporations.
ACHIEVINGASUSTAINABLEHYPERPRODUCTIVESTATE
The key to entering a hyperproductive state was not just the Scrum organizational pattern. It was acombinationof(1)theskilloftheteam,(2)theflexibilityofaSmalltalkdevelopmentenvironment,(3)theimplementation of what are now know as XP engineering practices, and (4) the way we systematicallystimulatedproductionprototypesthatrapidlyevolvedintoadeliverableproduct.
Furthermore,inthehyperproductivestate,theinitialScrumenteredwhatprofessionalathletesandmartialartistscall"thezone."Nomatterwhathappenedorwhatproblemsarose,theresponseoftheteamalwayswasfarbetterthantheresponseofanyindividual.ItwasreminiscentoftheCelticsbasketballteamattheirpeak,when they coulddonowrong. The impact of entering the zonewasnot just hyperproductivity. Peoplespersonalliveswerechanged.Teammemberssaidtheywouldneverforgetworkingontheproject,andthey
ãJeffSutherland1993-2021 90
wouldalwaysbelookingforanotherexperiencelikeit.Itinducedopen,teamoriented,fun-lovingbehaviorinunexpectedpersons.Thoseindividualswhocouldnotfunctionwellinanopen,hyperproductiveenvironmentself-selectedthemselvesoutoftheteambyfindingotherjobs.Thisreinforcedpositiveteambehaviorsimilarto biological systems, which select for fitness to the environment, resulting in improved performance ofindividualorganisms.
VMARK:THEFIRSTSENIORMANAGEMENTSCRUMWhen Easel Corporationwas acquired by VMARK (subsequenctly Informix, now Ascension Software) theoriginal Scrum team continued itswork on the same product. The VMARK seniormanagement teamwasintriguedbyScrumandaskedmetorunaweeklyseniormanagementteamScrumtodriveallthecompany’sproductstotheInternet.Thesemeetingsstartedin1995,andwithinafewmonths,theteamhadcausedtheintroductionoftwonewInternetproductsandrepositionedcurrentproductsasInternetapplications.SomemembersofthisteamleftVMARKtobecomeinnovatorsinemergingInternetcompanies,soScrumhadanearlyimpactontheInternet.
ItwasalsoatVMARKthatKenSchwaberwasintroducedtoScrum.KenandIhadworkedtogetheronandoffforyears.IshowedhimScrumandheagreeditworkedbetterthatotherprojectmanagementapproachesandwas similar tohowhebuilt projectmanagement software inhis company.Hequickly sold off theprojectmanagementsoftwarebusinessandworkedonbringingScrumtothesoftwareindustryatlarge.HisworkhashadanincredibleeffectondeployingScrumworldwide.
INDIVIDUAL:THEFIRSTINTERNETSCRUMInthespringof1996,IreturnedtoIndividual,Inc.,acompanyIco-foundedasVPofEngineeringin1988.MuchoftheScrumexperienceatIndividualhasbeendocumentedbyKenSchwaber[18].ThemostimpressivethingtomeaboutScrumatIndividualwasnotthattheteamdeliveredtwonewInternetproducts–andmultiplereleasesofoneoftheproducts–inasinglequarter.ItwasthefactthatScrumeliminatedseveralhoursadayofseniormanagementmeeting timestarting thedaythatScrumbegan,withinaweekofmyarrivalat thecompany. Because Individual had just gone public at the beginning of the Internet explosion, thereweremultiplecompetingprioritiesandconstantrevisionofmarketstrategy.Asaresult,thedevelopmentteamwasconstantly changingpriorities andunable todeliverproduct.Themanagement teamwasmeetingdaily todeterminestatusofprioritiesthatwerevieweddifferentlybyeverymanager.ThesemeetingswereeliminatedandtheScrummeetingsbecamethefocusforalldecisionmaking.
ItwasincrediblyproductivetoforcealldecisionstooccurinthedailyScrummeeting.Ifanyonewantedtoknowthestatusofspecificprojectdeliverablesorwantedtoinfluenceanypriority,heorshecouldonlydoitinthedailyScrummeeting.IremembertheseniorVPofmarketingsatinoneverymeetingforacoupleofweekssharingherdesperateconcernaboutmeetingInternetdeliverablesandtimetables.Theeffectontheteamwasnot to immediatelyrespondtoherdespair.Overaperiodof twoweeks, the teamself-organizedaroundaplantomeetherprioritieswithachievabletechnicaldeliverydates.Whensheagreedtotheplan,shenolongerhadtoattendanyScrummeetings.TheScrumreportedstatusontheWebwithgreenlights,yellow
ãJeffSutherland1993-2021 91
lights,andredlightsforallpiecesoffunctionality.Inthisway,theentirecompanyknewstatusinrealtime,allthetime.ThistransparencyofinformationhasbecomeakeycharacteristicofScum.
IDXSYSTEMS:THEFIRSTSCRUMINTHELARGEDuringthesummerof1996,IDXSystemshiredmeasseniorVPofengineeringandproductdevelopment.IDXhadover4,000customersandwasoneof the largestUShealthcaresoftwarecompanies,withhundredsofdevelopers working on dozens of products. Here was an opportunity to extend Scrum to large-scaledevelopment.
TheapproachatIDXwastoturntheentiredevelopmentgroupintoaninterlockingsetofScrums.Everypartoftheorganizationwasteambasedincludingthemanagementteam,whichincludedtwovicepresidents,aseniorarchitect,andseveraldirectors.Front-lineScrumsmetdaily.AScrumofScrums,whichincludedtheteamleadersofeachScruminaproductline,metweekly,ThemanagementScrummetmonthly.
ThekeylearningatIDXwasthatScrumscalestoanysize.Withdozensofteamsinoperation,themostdifficultproblemwasensuringthequalityoftheScrumprocessineachteam,particularlywhentheentireorganizationhadtolearnScrumallatonce.IDXwaslargeenoughtobringinproductivityexpertstomonitorthroughputoneveryproject.Whilemostteamswereonlyabletodoubletheindustryaverageinfunctionpointspermonthdelivered,severalteamsmovedintoahyperproductivestate,producingdeliverablefunctionalityatfourtofivetimestheindustryaverage.Theseteamsbecameshiningstarsintheorganizationandexamplesfortherestoftheorganizationtofollow.
One of the most productive teams at IDX was the Web Framerwork team that built a web frontendinfrastructure for all products. The infrastructure was designed to host all IDX applications, as well asseamlesslyinteroperatewithenduserorthirdpartyapplications.Itwasadistributedteamwithdevelopersin Boston, Seattle, and Vermont who met by teleconference in a daily Scrum meeting. The geographictransparencyof thismodelproduced thesamehighperformanceas co-located teamsandhasbecome thesignatureofhyperproductivedistributed/outsourcedScrums[29].
PATIENTKEEPERSCRUM:THEFIRSTENTERPRISESCRUM
In early 2000, I joinedPatientKeeper, Inc. as chief technology officer and began introducing Scrum into astartupcompany. Iwas the21stemployee,andwegrewthedevelopment teamfromadozenpeople to45peopleinsixmonths.PatientKeeperdeploysmobiledevicesinhealthcareinstitutionstocaptureandprocessfinancialandclinicaldata.Servertechnologysynchronizesthemobiledevicesandmovesdatatoandfrommultipleback-endlegacysystems.Arobusttechnicalarchitectureprovidesenterpriseapplicationintegrationto hospital and clinical systems. Data is forward-deployed from these systems in a PatientKeeper clinical
ãJeffSutherland1993-2021 92
repository.Servertechnologiesmigratechangesfromourclinicalrepositorytoacacheandthentodatastorageonthemobiledevice.PatientKeeperprovesthatScrumworksequallywellacrosstechnologyimplementations.
ThekeylearningatPatientKeeperhasinvolvedtheintroductionofeXtremeProgrammingtechniquesasawayto implementcodedeliveredbyaScrumorganization.Whileall teamsseemto find iteasyto implementaScrumorganizationalprocess,theydonotalwaysfinditeasytointroduceXP.Wewereabletodosometeamprogrammingandconstanttestingandrefactoring,particularlyaswemigratedalldevelopmenttoJavaandXML.ItwasmoredifficulttointroducetheseideaswhendeveloperswereworkinginCandC++.Afterayearof Scrummeetings in all areas of development, processesmatured enough to capitalize on Scrumprojectmanagementtechniques,whichwerefullyautomated.
CompleteautomationandtransparencyofdataallowedPatientKeepertomultithreadSprintsthroughmultipleteams.ThatincombinationwithimplementingaMetaScrumofseniorstakeholdersinthecompanyallowedPatientKeeper to run from top to bottom as a Scrum and become the first Scrum company to enter thehyperproductivestate,deliveringover45productionreleasesayearofalargeenterprisesoftwareplatform.ThisbecametheprototypefortheAll-at-Once,orTypeCScrum,implementedinatleastfivecompaniesby2006[41].
CONCLUSIONS
AfterintroducingScrumintofivedifferentcompaniesofdifferentsizesandwithdifferenttechnologies,IcanconfidentlysaythatScrumworksinanyenvironmentandcanscaleintoprogramminginthelarge.Inallcases,itwillradicallyimprovecommunicationanddeliveryofworkingcode.ThenextchallengeforScrum,inmyview,istoprovideatightintegrationoftheScrumorganizationalpatternwithXPprogrammingtechniques,combined with innovative approaches to distributed teams and stimulation of rapid software systemevolution.IbelievethesestrategiescangenerateahyperproductiveScrumonapredictablebasis.ThefirstScrumdidthisintuitivelyandthatwasitskeytoextremeperformanceandalife-changingexperience.
ãJeffSutherland1993-2021 93
DISTRIBUTEDSCRUM:AGILEPROJECTMANAGEMENTWITHOUTSOURCEDDEVELOPMENTTEAMS
JeffSutherland,Ph.D.AntonViktorov JackBlountNikolaiPuntikov
PatientkeeperStarSoftDev.LabsSirsiDynixStarSoftDev.Labs
Newton,MA,USSt.Petersburg,RussiaProvo,UT,USASt.Petersburg,Russia
[email protected]@[email protected]@starsoftlabs.com
ABSTRACTAgile projectmanagement with Scrum derives from best business practices in companies like Fuji-Xerox,Honda,Canon,andToyota.Toyotaroutinelyachievesfourtimestheproductivityand12timesthequalityofcompetitors. Can Scrumdo the same for globally distributed teams? TwoAgile companies, SirsiDynix andStarSoftDevelopmentLaboratoriesachievedcomparableperformancedevelopingaJavaapplicationwithover1,000,000linesofcode.During2005,adistributedteamof56ScrumdevelopersworkingfromProvo,Utah;Waterloo,Canada;andSt.Petersburg,Russia,delivered671,688linesofproductionJavacode.At15.3functionpoints per developer/month, this is the most productive Java project ever documented. SirsiDynix bestpracticesaresimilartothoseobservedondistributedScrumteamsatIDXSystems,radicallydifferentthanthosepromotedbyPMBOK,andcounterintuitivetopracticesadvocatedbysomeScrumtraining.ThispaperanalyzesandrecommendsbestpracticesforgloballydistributedAgileteams.
INTRODUCTIONScrumisanAgilesoftwaredevelopmentprocessdesignedtoaddenergy,focus,clarity,andtransparencytoprojectteamsdevelopingsoftwaresystems.Itleveragesartificialliferesearch[6]byallowingteamstooperateclosetotheedgeofchaostofosterrapidsystemevolution.Itcapitalizesonrobotsubsumptionarchitectures[5]byenforcingasimplesetofrulesthatallowsrapidself-organizationofsoftwareteamstoproducesystemswithevolvingarchitectures.AproperlyimplementedScrumwasdesignedtoincreasespeedofdevelopment,align individual andorganizationobjectives, create a culturedrivenbyperformance, support shareholdervalue creation, achieve stable and consistent communication of performance at all levels, and enhanceindividualdevelopmentandqualityoflife.
ScrumforsoftwaredevelopmentteamsbeganatEaselCorporation in1993andwasusedtobuildthe firstobject-oriented design and analysis (OOAD) tool that incorporated round-trip engineering. In a Smalltalkdevelopmentenvironment,codewasauto-generatedfromagraphicdesigntoolandchangestothecodefromtheSmalltalkintegrateddevelopmentenvironment(IDE)wereimmediatelyreflectedbackintodesign.
Adevelopmentprocesswasneededtosupportenterpriseteamswherevisualizationofdesignimmediatelygeneratedworkingcode.Thisledtoanextensivereviewofthecomputerscienceliteratureanddialoguewithleadersofhundredsofsoftwaredevelopmentprojects.KeyfactorsthatinfluencedtheintroductionofScrumatEaselCorporationwerefundamentalproblemsinherentinsoftwaredevelopment:
ãJeffSutherland1993-2021 94
• Uncertainty is inherent and inevitable in software development processes and products - Ziv’sUncertaintyPrinciple[54]
• Foranewsoftwaresystemtherequirementswillnotbecompletelyknownuntilaftertheusershaveusedit-Humphrey’sRequirementsUncertaintyPrinciple[58]
• Itisnotpossibletocompletelyspecifyaninteractivesystem–Wegner’sLemma[55]
• Ambiguous and changing requirements, combined with evolving tools and technologies makeimplementationstrategiesunpredictable.
• “All-at-Once” models of software development uniquely fit object-oriented implementation ofsoftware and help resolve these challenges. They assume the creation of software involvessimultaneouslyworkonrequirements,analysis,design,coding,andtesting,thendeliveringtheentiresystemallatonce[32].
“ALL-AT-ONCE”DEVELOPMENTMODELSThesimplest“All-at-Once”model isasinglesuper-programmercreatinganddeliveringanapplicationfrombeginning to end. This can be the fastest way to deliver a product that has good internal architecturalconsistencyandisthe“hacker”modelofimplementation.Forexample,ina“skunkworks”projectpriortothefirstScrum,asingleindividualsurroundedbyasupportteamspenttwoyearswritingeverylineofcodefortheMatisseobjectdatabase[59]usedtodrive$10Bnuclearreprocessingplantsworldwide.Atlessthan50,000linesofcode,thenuclearengineerssaiditwasthefastestandmostreliabledatabaseeverbenchmarkedfornuclearplants.
IBMdocumentedavariantofthisapproachcalledtheSurgicalTeamandconsidereditthemostproductiveapproachtosoftwaredevelopment[33].TheSurgicalTeamconcepthasafatalflawinthatthereareatmostoneortwoindividualseveninalargecompanythatcanexecutethismodel.Forexample,ittookthreeyearsforacompetentteamofdeveloperstounderstandtheconceptualeleganceoftheMatisseobjectserverwellenoughtomaintainit.Thesingle-programmermodeldoesnotscalewelltolargeprojects.
Thenextlevelof“All-at-Once”developmentishandcuffingtwoprogrammerstogether.Pairprogramming,aneXtremeProgrammingpractice[24],isanimplementationofthis.Here,twodevelopersworkingatthesameterminaldeliveracomponentofthesystemtogether.Thishasbeenshowntodeliverbettercode(usability,maintainability,flexibility,extendibility)fasterthantwodevelopersworkingindividually[34].Thechallengeistoachieveasimilarproductivityeffectwithmorethantwopeople.
Scrum,ascalable, team-based“All-at-Once”model,wasmotivatedbythe Japaneseapproachtoteam-basednew product development combined with simple rules to enhance team self-organization (see Brooks’subsumptionarchitecture[5]).AtEasel,thedevelopmentteamwasalreadyusinganiterativeandincrementalapproach to building software [35]. Features were implemented in slices where an entire piece of fullyintegrated functionalityworked at the end of an iteration.What intrigued uswas Takeuchi andNonaka’sdescriptionoftheteam-buildingprocessforsettingupandmanagingaScrum[1].Theideaofbuildingaself-empoweredteaminwhichadailyglobalviewoftheproductcausedtheteamtoself-organizeseemedlikethe
ãJeffSutherland1993-2021 95
rightidea.Thisapproachtomanagingtheteam,whichhadbeensosuccessfulatHonda,Canon,andFujitsu,alsoresonatedwithresearchonsystemsthinkingbyProfessorSengeatMIT[16].
HYPERPRODUCTIVITYINSCRUMScrumwasdesignedtoallowaveragedeveloperstoself-organizeintohighperformanceteams.ThefirstScrumachieved ahyperproductive state in1993-1994becauseof threeprimary factors.The firstwas the Scrumprocessitself,characterizedby15minutedailymeetingswhereeachpersonanswersthreequestions–whatdidyouaccomplishyesterday,whatwillyoudotoday,andwhatimpedimentsaregettinginyourway?ThisisnowpartofthestandardScrumorganizationalpattern[26].Second,theteamimplementedallXPengineeringprocesses[24]includingpairprogramming,continuousbuilds,andaggressiverefactoring.Andthird,theteamsystematicallystimulatedrapidevolutionofthesoftwaresystem.
OneofthemostinterestingcomplexityphenomenaobservedinthefirstScrumwasa“punctuatedequilibrium”effect[36].Thisphenomenonoccursinbiologicalevolutionwhenaspeciesisstableforlongperiodsoftimeandthenundergoesasuddenjumpincapability.DannyHillissimulatedthiseffectonanearlysuper-computer,theConnectionMachine[60].
“TheartificialorganismsinHillis’sparticularworldevolvednotbysteadyprogressofhillclimbingbutbythesuddenleapsofpunctuatedequilibrium…WithartificialorganismsHillishadthepowertoexamineandanalyzethe genotype as easily as the realizedphenotypes…While thepopulation seemed tobe restingduring theperiodsofequilibrium…theunderlyinggeneticmakeupwasactivelyevolving.Thesuddenincreaseinfitnesswasnomoreaninstantoccurrencethantheappearanceofanewbornindicatessomethingspringingoutofnothing; thepopulation seemed tobe gestating itsnext jump. Specifically, the genepool of thepopulationcontainedasetofepistaticgenesthatcouldnotbeexpressedunlessallwerepresent;otherwisetheallelesforthesegeneswouldberecessive.”[61]
UsingScrumwithafullyintegratedcomponentdesignenvironmentleadstounexpected,rapidevolutionofasoftware system with emergent, adaptive properties resembling the process of punctuated equilibrium.Sudden leaps in functionality resulted in earlier than expected delivery of software in the first Scrum.Developmenttasks,originallyplannedtotakedays,couldoftenbeaccomplishedinhoursusingsomeoneelse’scodeasastartingpoint.
This aspect of self-organization is now understood as a type of Set-Based Concurrent Engineering (SBCE)practiced at Toyota [57]. Developers consider sets of possible solutions and gradually narrow the set ofpossibilitiestoconvergeonafinalsolution.Decisionsonhowandwheretoimplementafeatureisdelayeduntilthelastpossiblemoment.Themostevolvedcomponentisselected“justintime”toabsorbnewfunctionality,resultinginminimalcodingandamoreelegantarchitecture.Thusemergentarchitecture,acoreprincipleinallAgile processes, is not randomevolution. Properly implemented, it is an SBCE technique viewed as a bestbusinesspracticeinsomeoftheworld’sleadingcorporations.
ãJeffSutherland1993-2021 96
DISTRIBUTED,OUTSOURCEDSCRUM
Scrumwasdesignedtoachieveahyperproductivestatewhereproductivityincreasesbyanorderofmagnitudeoverindustryaverages.Manysmall,collocatedteamshaveachievedthiseffect.Thequestionforthispaperiswhetheralarge,distributed,outsourcedteamcanachievethehyperproductivestate.
U.S.,European,orJapanesecompaniesoftenoutsourcesoftwaredevelopmenttoEasternEurope,Russia,ortheFar East. Typically, remote teams operate independently and communication problems limit productivity.While there isa largeamountofpublishedresearchonprojectmanagement,distributeddevelopment,andoutsourcingstrategiesasisolateddomains,therearefewdetailedstudiesofbestprojectmanagementpracticesonlargesystemsthatarebothdistributedandoutsourced.
CurrentrecommendedScrumpracticeisforlocalScrumteamsatallsitestosynchronizeonceadayviaaScrumof Scrums meeting. Here we describe something rarely seen. At SirsiDynix, all Scrum teams consist ofdevelopersfrommultiplesites.WhilesomeAgilecompanieshavecreatedgeographicallytransparencyonasmall scale, SirsiDynixuses fully integratedScrum teamswithover50developers in theU.S., Canada, andRussia.ThisstrategyhelpedbuildanewimplementationofplatformandsystemarchitectureforacomplexIntegratedLibrarySystem(ILS).TheILSsystemissimilartoaverticalmarketERPsystemwithapublicportalinterfaceusedbymorethan200millionpeople.
Best practices for distributed Scrum seen on this project consist of (1) daily Scrum teammeetings of alldevelopersfrommultiplesites,(2)dailymeetingsofProductOwnerteam(3)hourlyautomatedbuildsfromone central repository, (4)nodistinctionbetweendevelopers at different sites on the same team, (5) andseamless integrationofXPpractices likepairprogrammingwithScrum.While similarpracticeshavebeenimplementedonsmalldistributedScrumteams[40]this isthefirstdocumentedprojectthatdemonstratesScrumhyperproductivityforlargedistributed/outsourcedteamsbuildingcomplexenterprisesystems.
DISTRIBUTED TEAM MODELS
HereweconsiderthreedistributedScrummodelscommonlyobservedinpractice:
• IsolatedScrums-Teamsareisolatedacrossgeographies.Inmostcasesoff-shoreteamsarenotcross-functionalandmaynotbeusingtheScrumprocess.
ãJeffSutherland1993-2021 97
• DistributedScrumofScrums–ScrumteamsareisolatedacrossgeographiesandintegratedbyaScrumofScrumsthatmeetsregularlyacrossgeographies.
• Totally Integrated Scrums – Scrum teams are cross-functional with members distributed acrossgeographies.IntheSirsiDynixcase,theScrumofScrumswaslocalizedwithallScrumMastersinUtah.
MostoutsourceddevelopmenteffortsuseadegenerativeformoftheIsolatedScrumsmodelwhereoutsourcedteamsarenotcross-functionalandnotAgile.RequirementsmaybecreatedintheU.S.anddevelopedinDubai,ordevelopmentmayoccurinGermanyandqualityassuranceinIndia.Typically,cross-culturalcommunicationproblemsarecompoundedbydifferencesinworkstyleintheprimaryorganizationvs.theoutsourcedgroup.Intheworstcase,outsourcedteamsarenotusingScrumandtheirproductivityistypicalofwaterfallprojectsfurtherdelayedbycross-continentcommunicationslagtime.ImplementationsofScruminadatarichCMMILevel5companysimultaneouslyrunningwaterfall,incremental,anditerativeprojects,showedproductivityofScrumteamswasat leastdoublethatofwaterfall teams,evenwithCMMILevel5reportingoverhead[62].OutsourcedteamsnotusingScrumwilltypicallyachievelessthanhalfthevelocityoftheprimarysiteusingScrum.
Figure11:StrategiesfordistributedScrumteams[12].
ThelatestthinkingintheProjectManagementInstituteGuidetotheProjectManagementBodyofKnowledge(PMBOK) models is a degenerative case of isolated non-Scrum teams [63]. This is a spiral waterfallmethodologywhichlayerstheUnifiedModelingLanguage(UML)andtheRationalUnifiedProcess(RUP)ontoteamswhicharenotcross-functional[64].Itpartitionsworkacrossteams,createsteamswithsilosofexpertise,andincorporatesaphasedapproachladenwithartifactsthatviolatetheprinciplesofleandevelopment[13].
Best practice is aDistributed Scrumof Scrumsmodel. Thismodel partitionswork across cross-functional,isolated Scrum teamswhile eliminatingmost dependencies between teams. Scrum teams are linked by aScrum-of-ScrumswhereScrumMasters(teamleaders/projectmanagers)meetregularlyacrosslocations.This
Isolated Scrums Teams
Distributed Scrum of Scrums
Integrated Scrums
ãJeffSutherland1993-2021 98
encourages communication, cooperation, and cross-fertilization and is appropriate fornewcomers toAgiledevelopment.
AnIntegratedScrumsmodelhasallteamsfullydistributedandeachteamhasmembersatmultiplelocations.Whilethisappearstocreatecommunicationandcoordinationburdens,thedailyScrummeetingshelptobreakdownculturalbarriersanddisparitiesinworkstyles.Onlargeenterpriseimplementations,itcanorganizetheproject into a single whole with an integrated global code base. Proper implementation of this approachprovideslocationtransparencyandperformancecharacteristicssimilartosmallco-locatedteams.Asmaller,butsimilar,distributedteamatIDXSystemsCorporationduring1996-2000achievedalmosttentimesindustryaverage performance [40]. The SirsiDynix model approached this level of performance fordistributed/outsourced Integrated Scrums. It appears to be the most productive distributed team everdocumentedforalargeJavaenterprisesystemwithmorethanonemillionlinesofcode.ThisIntegratedScrumsmodelisrecommendedforexperiencedAgileteamsatmultiplelocations.
SIRSIDYNIX CASE STUDY SirsiDynixhasapproximately4,000libraryandconsortiaclients,servingover200millionpeoplethroughover20,000libraryoutletsintheAmericas,Europe,Africa,theMiddleEastandAsia-Pacific.JackBlount,PresidentandCEOofDynixandnowCTOofthemergedSirsiDynixcompany,negotiatedanoutsourceagreementwithStarSoft who staffed the project with over 20 qualified engineers in 60 days. Significant developmentmilestoneswerecompletedinafewweeksandjointdevelopmentprojectsareefficientlytrackedandcontinuetobeonschedule.
StarSoft Development Labs, Inc. is a software outsourcing service provider in Russia and Eastern Europe.HeadquarteredinCambridge,Massachusetts,USA,StarSoftoperatesdevelopmentcenters inSt.Petersburg,Russia and Dnepropetrovsk, Ukraine, employing over 450 professionals. StarSoft has experience handlingdevelopmenteffortsvaryinginsizeanddurationfromjustseveralengineersworkingforafewmonthstolarge-scale projects involving dozens of developers and spanning several years. StarSoft successfully uses AgiledevelopmentandparticularlyXPengineeringpracticestomaintainCMMLevel3certification.
HIDDEN COSTS OF OUTSOURCING Thehiddencostsofoutsourcingaresignificant,beginningwithstartupcosts.Barthelemy[65]surveyed50companiesandfoundthat14%ofoutsourcingoperationswerefailures.Intheremainder,costsoftransitioningtoanewvendoroftencanceledoutanticipatedsavingsfromlowlaborcosts.Theaveragetimefromevaluatingoutsourcingtobeginningofvendorperformancewas18monthsforsmallprojects.Asaresult,theMITSloanManagementReviewadvisesreadersnottooutsourcecriticalITfunctions.
TheGermanInstituteforEconomicResearchanalyzed43,000Germanmanufacturingfirmsfrom1992-2000andfoundthatoutsourcingservicesledtopoorcorporateperformance,whileoutsourcingproductionhelped
ãJeffSutherland1993-2021 99
[66].Whilethisisamanufacturingstudyratherthansoftwaredevelopment,itsuggeststhatoutsourcingcoredevelopmentmayprovidegainsnotseenotherwise.
FIGURE12-SIRSIDYNIXLINESOFNEWJAVACODEINTHOUSANDSFROM2003-2006.
Largesoftwareprojectsareveryhighrisk.The2003StandishChaosReportshowsuccessratesofonly34%.51%ofprojectsareoverbudgetorlackingcriticalfunctionality.15%aretotalfailures[67].
SirsiDynixsidesteppedmanyofthehiddencosts,directlyoutsourcedprimaryproduction,andusedIntegratedScrumstocontrolrisk.Thegoalsofincreasingoutputperteammemberandlinearlyincreasingoveralloutputby increasing teamsizewereachieved.Productionvelocitymore thandoubledwhen the30personNorthAmericandevelopmentteamwasaugmentedwith26RussiansfromStarSoftinDecember2005.
INTENTOFTHEINTEGRATEDSCRUMSMODELAnAgile companybuildinga largeproductand facing time-to-marketpressureneeds toquicklydoubleorquadrupleproductivitywithinaconstrainedbudget.Thelocaltalentpoolisnotsufficienttoexpandteamsizeandsalarycostsaremuchhigherthanoutsourcedteams.Ontheotherhand,outsourcingisonlyasolutionifAgilepracticesareenhancedbycapabilitiesoftheoutsourcedteams.Theprimarydriverisenhancedtechnicalcapabilityresultingindramaticallyimprovedthroughputofnewapplicationfunctionality.Costsavingsareasecondarydriver.
CONTEXT
ãJeffSutherland1993-2021 100
Softwarecomplexityanddemands for increased functionalityareexponentially increasing inall industries.WhenanauthorofthispaperflewF-4aircraft incombatin1967,8%ofpilotfunctionsweresupportedbysoftware. In 1982, the F16 software support was 45%, and by 2000, the F22 augmented 80% of pilotcapabilitieswithsoftware[63].Demandsforeaseofuse,scalability,reliability,andmaintainabilityincreasewithcomplexity.
SirsiDynixwasconfrontedwiththerequirementtocompletelyre-implementalegacylibrarysystemwithover12,500installedsites.Largeteamsworkingovermanyyearsinachangingbusinessenvironmentfacedmanynewrequirementsinthemiddleoftheproject.Tocomplicatemattersfurther,thelibrarysoftwareindustrywas in a consolidating phase. Dynix started the project in 2002 and merged with Sirsi in 2005 to formSirsiDynix.
Fortunately,DynixstartedwithascalableAgileprocessthatcouldadapttochangingrequirementsthroughouttheproject.Timetomarketdemandedmorethandoublingofoutput.ThatcouldonlyhappenbyaugmentingresourceswithAgileteams.StarSoftwasselectedbecauseoftheirhistoryofsuccessfulXPimplementationsandtheirexperiencewithsystemslevelsoftware.
The combinationof high risk, large scale, changingmarket requirements,merger and acquisitionbusinessfactors,andtheSirsiDynixexperiencewithScrumcombinedwithStarSoftsuccesswithXPledthemtochooseanIntegratedScrumsimplementation. JackBlount'spastexperiencewithAgiledevelopmentprojectsatUSDataAuthority,TeleComputingandJDEdwardswherehehadusedIsolatedScrumsandDistributedScrumofScrumsmodelsdidnotmeethisexpectations.ThiswasakeyfactorinhisdecisiontostructuretheprojectasIntegratedScrums.
FORCES
COMPLEXITYDRIVERSThe Systems and Software Consortium (SSCI) has outlined drivers, constraints, and enablers that forceorganizationstoinvestinreal-timeprojectmanagementinformationsystems.ScalableScrumimplementationswithminimaltoolingareoneofthebestreal-timeinformationgeneratorsinthesoftwareindustry.
SSCIcomplexitydriversaredescribedas[63]:
• Increasingproblemcomplexityshiftingfocusfromrequirementstoobjectivecapabilitiesthatmustbemetbylargerteamsandstrategicpartnerships.
• Increasing solution complexity which shifts attention from platform architectures to enterprisearchitecturesandfullyintegratedsystems.
ãJeffSutherland1993-2021 101
• Increasingtechnicalcomplexityfromintegratingstandalonesystemstointegratingacrosslayersandstacksofcommunicationsandnetworkarchitectures.
• Increasingcompliancecomplexityshiftingfromproprietarytoopenstandards.
• Increasingteamcomplexityshiftingfromasingleimplementertostrategicteamingandmergersandacquisitions.
SirsiDynixfacedalloftheseissues.Legacyproductsweredifficulttoselltonewcustomers.Theyneededanewproductwithcomplete functionality for the libraryenterprisebasedonnew technologies thatwerehighlyscalable,easilyexpandable,andusedthelatestcomputerandlibrarystandards,
The SirsiDynix Horizon 8.0 architecture supports a wide variety of users from publication acquisition tocataloging,searching,reserving,circulating,orintegratinginformationfromlocalandexternalresources.Thedecisionwasmade touse Javawith J2EE,amodulardesign,database independency,maximumuseof freeplatformsandtools,andwidesupportofMARC21,UNIMARC,Z39.50andotherILSstandards.
Theprojectusesathree-tierarchitectureandHibernateasadatabaseabstractionlayer.Oracle10g,MSSQL,andIBMDB2supportisprovided.TheJBoss4ApplicationserverisusedwithaJavaGUIClientwithWebStartbootstrap. It is a cross-platformproduct supportingMSWindows2000,XP, 2003,RedHatLinux, andSunSolaris. Built-in,multi-language support has on-the-fly resource editing for ease of localization. Other keytechnologiesareJAAS,LDAP,SSL,Velocity,Xdoclet,JAXB,JUnit,andJython.
TOPISSUESINDISTRIBUTEDDEVELOPMENTTheSSCIhascarefullyresearchedtopissuesindistributeddevelopment[63],allofwhichhadtobehandledbySirsiDynixandStarSoft.
• Strategic:Difficult leveragingavailableresources,bestpracticesareoftendeemedproprietary,aretimeconsuminganddifficulttomaintain.
• Projectandprocessmanagement:Difficultysynchronizingworkbetweendistributedsites.
• Communication:Lackofeffectivecommunicationmechanisms.
• Cultural:Conflictingbehaviors,processes,andtechnologies.
• Technical:Incompatibledataformats,schemas,andstandards.
• Security:Ensuringelectronictransmissionconfidentialityandprivacy.
ãJeffSutherland1993-2021 102
TheuniquewayinwhichSirsiDynixandStarSoftimplementedanIntegratedScrumsmodelcarefullyaddressedalloftheseissues.
SOLUTION:INTEGRATEDSCRUMSTherearethreerolesinaScrum:theProductOwner,theScrumMaster,andtheTeam.SirsiDynixusedtheserolestosolvethestrategicdistributionproblemofbuildingahighvelocity,real-timereportingorganizationwithanopensourceprocessthatiseasytoimplementandlow-overheadtomaintain.
Forlargeprograms,aChiefScrumMastertorunaScrumofScrumsandaChiefProductOwnertocentrallymanage a single consolidated andprioritizedproduct backlog is essential. SirsiDynix located the ScrumofScrumsandtheProductOwnerteamsinUtah.
TEAMFORMATIONThe second major challenge for large projects is process management, particularly synchronizing workbetweensites.ThiswasachievedbysplittingteamsacrosssitesandfinetuningdailyScrummeetings.
FIGURE 3 SCRUM TEAMS SPLIT ACROSS SITES. PO=PRODUCT OWNER, SM=SCRUMMASTER,TLD=TECHNICALLEAD.
TeamsatSirsiDynixweresplitacrossthefunctionalareasneededforanintegratedlibrarysystem.HalfofaScrumteamistypicallyinProvo,Utah,andtheotherhalfinSt.Petersburg.Thereareusually3-5peopleontheUtahpartoftheteamand4ormoreontheSt.Petersburgportionoftheteam.TheSearchandReportingTeamsaresmaller.TherearesmallernumbersofteammembersinSeattle,Denver,St.Louis,andWaterloo,Canada.
SCRUM MEETINGS Teamsmeet across geographies at 7:45am Utah timewhich is 17:45 St. Petersburg time. Teams found itnecessary to distribute answers to the three Scrum questions in writing before the Scrummeeting. Thisshortensthetimeneededforthejoinmeetingteleconferenceandhelpsovercomeanylanguagebarriers.Eachindividualreportsonwhattheydidsincethelastmeeting,whattheyintendtodonext,andwhatimpedimentsareblockingtheirprogress.
SM
Dev
Dev
Dev
TLdCatalogue Serials Circulation Search Reporting
StarSoft
SirsiDynix
PO PO PO
ãJeffSutherland1993-2021 103
EmailexchangeonthethreequestionsbeforethedailyScrumteleconferencewasusedthroughouttheprojecttoenablephonemeetingstoproceedmoresmoothlyandefficiently.ThesedailyteamcallshelpedthepeopleinRussiaandtheU.S.learntounderstandeachother.Incontrast,mostoutsourceddevelopmentprojectsdonotholdformaldailycallsandthecommunicationbridgeisneverformed.
Figure 4 – Scrum Team meetings
Localsub-teamshaveanadditionalstandupmeetingatthebeginningofthedayinSt.Petersburg.Everyoneusesthesameprocessandtechnologiesanddailymeetingscoordinateactivitieswithintheteams.
ScrumMasters are all in Provo,Utah orWaterloo, Canada, andmeet in a Scrumof Scrums everyMondaymorning.Hereworkiscoordinatedacrossteams.ArchitectsaredirectlyallocatedtoproductionScrumteamsandalllocatedinUtah.AnArchitecturegroupalsomeetsonMondayaftertheScrumofScrumsmeetingandcontrolsthedirectionoftheprojectarchitecturethroughtheScrummeetings.AProductOwnerresidentinUtahisassignedtoeachScrumteam.AchiefProductOwnermeetsregularlywithallProductOwnerstoassurecoordinationofrequirements.
SirsiDynixachievedstrongcentralcontrolofteamsacrossgeographiesbycentrallylocatingScrumMasters,ProductOwners,andArchitects.Thishelpedthemgetconsistentperformanceacrossdistributedteams.
SPRINTS SprintsaretwoweekslongontheSirsiDynixproject.ThereisaSprintplanningmeetingsimilartoanXPreleaseplanningmeeting inwhichrequirements fromUserStoriesarebrokendownintodevelopment tasks.MosttasksrequirealotofquestionsfromtheProductOwnersandsometaskstakemoretimethaninitialestimates.
7:45am Provo, Utah
St. Petersburg, Russia 17:45pm
Local Team Meeting
Scrum Team Meeting
ãJeffSutherland1993-2021 104
ThelagtimeforUtahProductOwnerresponsetoquestionsonUserStoriesforcesmultitaskinginSt.Petersburgandthisisnotanidealsituation.SometimesnewtasksarediscoveredafterqueryingProductOwnersduringtheSprintaboutfeaturedetails.
CodeisfeaturecompleteanddemoedattheendofeachSprint.Upuntil2006,ifitmettheProductOwner’sfunctionalrequirement,itwasconsidereddone,althoughfulltestingwasnotcompleted.Itwasnotdeliverablecodeuntil SirsiDynix strengthened its definitionof “done” to include all testing in2006.Allowingwork inprogresstocrossSprintboundariesintroduceswaittimesandgreaterriskintotheproject.Itviolatestheleanprincipleofreducingworkinprogressandincreasesrework.
PRODUCT SPECIFICATIONS RequirementsareintheformofUserStoriesusedinmanyScrumandXPimplementations.Someofthemarelengthyanddetailed,othersarenot.AlotofquestionsresultafterreceivingthedocumentinSt.PetersburgwhichareresolvedbyindailyScrummeetings,instantmessaging,oremail.
STORY FOR SIMPLE RENEWALS USE CASE: Patronbringsbookorotheritemtostafftoberenewed.
PatronJohnSmithcheckedout"TheDaVinciCode"thelasttimehewasinthelibrary.Todayheisbackinthelibrarytopickupsomethingelseandbrings"TheDaVinciCode"withhim.Hehandsittothestaffuserandasksforittoberenewed.Thestaffusersimplyscanstheitembarcodeatcheckout,andthesystemtreatsitasarenewalsincetheitemisalreadycheckedouttoJohn.Thischangestheloanperiod(extendstheduedate)forthelengthoftherenewalloan.Itemandpatroncirculationhistoryareupdatedwithanewrowshowingtherenewaldateandnewduedate.Countsdisplayforthenumberofrenewalsusedandremaining.TheitemisreturnedtoPatronJohnSmith.
Assumptions:
Itembeingrenewediscurrentlycheckedouttotheactivepatron
• Norequestsorreservationsoutstanding
• Itemwasnotoverdue
• Itemdoesnothaveaproblemstatus(lost,etc)
• Norenewmaximumshavebeenreached
• Noblock/circulationmaximumshavebeenreached
ãJeffSutherland1993-2021 105
• Patron'ssubscriptionsareactiveandnotwithinrenewalperiod
• Norenewalchargesapply
• Norecallsapply
RenewalisfromCheckOut(notCheckIn)
StaffUserhasrenewalprivileges
Verification(Howtoverifycompletion):
• LaunchCheckOut
• Retrieveapatronwhohasanitemalreadycheckedoutbutnotyetoverdue
• Enterbarcodeforcheckedoutitemintobarcodeentryarea(asifitisbeingcheckedout),andpress<cr>.
• Systemcalculatesnewduedateaccordingtocirculationrulesandagencyparameters.
• Therenewalcountisincremented(Staffrenewalwithitem)
• If user views "Circulation ItemDetails", the appropriate Renewals information should be updated(renewalsused/remaining)
• Cursor focus returns tobarcodeentryarea, ready to receivenext scan (ifpreviousbarcode is stilldisplayed,itshouldbeautomaticallyreplacedbywhateverisenterednext)
• Acheckoftheitemandpatroncirculationstatisticsscreensshowanewrowfortherenewalwiththerenewaldate/timeandthenewduedate.
Forthisproject,St.Petersburgstafflikesadetaileddescriptionbecausethesystemisacomprehensiveandcomplexsystemdesignedforspecializedlibrarians.Asaresult,thereisalotofknowledgethatneedstobeembeddedintheproductspecification.
ThewayslibrariesworkinSt.PetersburgareverydifferentthanEnglishlibraries.Russianlibrariesoperatelargelyviamanualoperations.WhileprocesseslooksimilartoEnglishlibrariesonthesurface,theunderlyingdetailsarequitedifferent.Therefore,userstoriesdonothavesufficientdetailforRussianprogrammers.
TESTING Developerswriteunittests.TheTestteamandProductOwnersdomanualtesting.AnAutomationTestteaminUtahcreatesscriptsforanautomatedtestingtool.Stresstestingisasneeded.
ãJeffSutherland1993-2021 106
During the Sprint, the Product Owner tests features that are in the Sprint backlog. Up until 2006, testersreceivedastableSprintbuildonlyaftertheSprintdemo.Thereasonforthiswasalowertester/developerratiothanrecommended.
Thereare30teammembersinNorthAmericaand26teammembersinSt.Petersburgonthisproject.TheSt.Petersburgteamhasoneprojectleader,3technicalteamleaders,18developers,1testlead,and3testers.Thislowtester/developerratioinitiallymadeitimpossibletohaveafullytestedpackageofcodeattheendoftheSprints.
Thetest-firstapproachwasinitiallyencouragedandnotmandated.Testswerewrittensimultaneouslywithcodemostofthetime.GUIswerenotunittested.
ComponentTestCases Tested
Acquisitions 529 384
Binding 802 646
Cataloging 3101 1115
Circulation 3570 1089
Common 0 0
ERM 0 0
PacSearching 1056 167
Serials 2735 1714
SubTotal 11793 5115
Figure5–TestCasesCreatedvs.Tested(codedandworking)
Inthesummerof2006,anewCTOofSirsiDynix,TalinBingham,tookovertheprojectandintroducedTestDrivenDesign.EverySprintstartswiththeusualSprintPlanningmeetingandteamsareresponsibleforwritingfunctionaltestsbeforedoinganycoding.Oncefunctionaltestsarewrittenandreviewed,codingstarts.Test-first coding is mandated. When coding is complete, developers run unit tests and manually pass all thefunctionaltestsbeforecheckinginchangestotherepository.
ãJeffSutherland1993-2021 107
FunctionalArea ReserveBookRoom
TaskDescription CheckthatitemsfromItemListisplacedunderReservewith“Inactive”status
Condition 1. UserhasrightforplacingItemsunderReserve
2. AtleastoneItemListexistsinthesystem
3. DefaultReserveItemStatusinSessionDefaultsissetto”Inactive”
EntryPoint Launcherisopened
TestData Nospecificdata
ACTION1. Reserve>ReserveItem
2. Select“ItemSearch”icon
3. Select“ItemList”intheComboboxlistofsearchoptionsandenterappropriateItemlistname
4. PressEnter
5. SelectallItemswhichappearintheItemSearchcomboboxandpress“OK”
EXPECTEDRESULTS1. ItemsthatwereinItemlistshouldappearinthe
listinReserveItem
2. Status of all items that has been just addedshouldbeshownas“Inactive”
3. Savebuttonshouldbeinactive
4. All corresponding Item should retain theiroriginalparameters
Figure6–FunctionalTestExample
AutomationtestingisdoneusingtheCompuwareTestPartnertool,butthereisstillroomforimprovementoftestcoverage.
CONFIGURATION MANAGEMENT SirsiDynixwasusingCVSassourcecoderepositorywhenthedecisionwasmadetoengageanoutsourcingfirm.Atthattime,SirsiDynixmadeadecisionthatCVScouldnotbeusedeffectivelybecauseoflackofsupportfordistributeddevelopment, largely seen in long code synchronization times.Other toolswere evaluated andPerforcewaschosenasthebestsolution.
ãJeffSutherland1993-2021 108
StarSofthadseenpositiveresultsonmanyprojectsusingPerforce. It is fast, reliableandoffers localproxyserversfordistributedteams.Althoughnotacheapsolution,ithasbeenveryeffectivefortheSirsiDynixproject.
Automatedbuildsruneveryhourwithemailgeneratedbacktodevelopers.Ittakes12minutestodoabuild,30minutesifthedatabasechanges.StarSoftwouldliketoseefasterbuildsandtrueconcurrentengineering.RightnowbuildsareonlystableeverytwoweeksatSprintboundaries.
PAIR PROGRAMMING, REFACTORING, AND OTHER XP PRACTICES StarSoftisanXPcompanyandtriestointroduceXPpracticesintoalltheirprojects.Pairprogrammingisdoneonmorecomplicatedpiecesoffunctionality.RefactoringwasplannedforfutureSprintsandnotdoneineveryiterationasinXP.Someradicalrefactoringwithoutlossoffunctionalityoccurredastheprojectapproachedcompletion.Continuousintegrationisimplementedashourlybuilds.Onthisproject,thesethreeengineeringpracticeswereusedwithScrumastheprimaryprojectmanagementmethodology.
MEASURING PROGRESS
TheprojectusestheJira<http://www.atlassian.com>issuetrackingandprojectmanagementsoftware.Thisgives everyone on the project a real-time view into the state of Sprints. It also provides comprehensivemanagement reporting tools. The Figure below shows the Sprint burn-down chart, a snapshot of EarnedBusiness,andasynopsisofbugstatus.
FIGURE6–SIRSIDYNIXHORIZON8.0PROJECTDASHBOARD
DatafromJiracanbedownloadedintoExceltocreateanyrequesteddataanalysis.Highvelocityprojectsneedanautomatedtooltotrackstatusacrossteamsandgeographies.Thebesttoolssupportbugtrackingandstatusofdevelopmenttasksinonesystemandavoidextraworkondataentrybydevelopers.Suchtoolsshouldtracktaskscompletedbydevelopersandworkremaining.Theyprovidemoredetailedandusefuldatathantimesheets,whichshouldbeavoided.Timesheetsareextraoverheadthatdonotprovideusefulinformationonthestateoftheproject,andarede-motivatingtodevelopers.
ãJeffSutherland1993-2021 109
OthercompanieslikePatientKeeper[68]havefoundtoolsthatincorporatebothdevelopmenttasksanddefectsthatcanbepackagedintoaSprintBacklogarehighlyusefulforcomplexdevelopmentprojects.ThousandsoftasksanddozensofSprintscanbeeasilymaintainedandreviewedreal-timewiththerighttool.
RESULTINGCONTEXTWITHINTEGRATEDSCRUMSCollaborationofSirsiDynixandStarSoftturnedtheHorizon8.0projectintooneofthemostproductiveScrumprojectseverdocumented.Forexample,dataisprovideinthetablebelowonaprojectthatwasdoneinitiallywithawaterfallteamandthenre-implementedwithaScrumteam[69].Thewaterfallteamtook9monthswith60peopleandgenerated54000 linesof code. Itwasre-implementedbyaScrumteamof4.5people in12months.Theresulting50,803linesofcodehadmorefunctionalityandhigherquality.
Scrum Waterfall SirsiDynix
PersonMonths 54 540 827
JavaLOC 50,803 54000 671,688
FunctionPoints 959 900 12673
FPperdev/month 17.8 2.0 15.3
FIGURE 7 – FUNCTION POINTS/DEVELOPER MONTH FOR COLLOCATED VS.DISTRIBUTEDPROJECTS.
CapersJonesofSoftwareProductivityResearchhaspublishedextensivetablesonaveragenumberoffunctionpointsperlinesofcodeforallmajorlanguages[70].SincetheaveragelinesofcodeperfunctionpointforJavais53,wecanestimatethenumberoffunctionpointsintheScrumapplication.Thewaterfallimplementationisknowntohavefewerfunctionpoints.
DistributedteamsworkingonHorizon8.0generated671,688 linesofcode in14.5monthswith56people.Duringthisperiodtheyradicallyrefactoredthecodeontwooccasionsandreducedthecodebasedby275,000.Theyhavenotbeenpenalizedforrefactoringasthatisrarelydoneinlargewaterfallprojectsinthedatabasefromwhich Capers derived his numbers. They have also not been rewarded for refactoring even thoughreducinglinesofcodeisviewedasimportantasaddingnewcodeonwell-runAgileprojects.
Joneshasalsoshownfromhisdatabaseoftensofthousandsofprojectsthatindustryaverageproductivityis12.5functionpointsperdeveloper/monthforaprojectof900functionpointsandthatthisdropsto3foraprojectwith13000functionpoints[7].Someofthisisdueto4GLandothercode-automationtoolsusedonsmallprojects,manyofwhicharenotimplementedinthirdgenerationlanguageslikeJava.
ãJeffSutherland1993-2021 110
TheSirsiDynixprojectisalmostasproductiveasthesmallScrumprojectwithacollocatedteamof4.5people.Foragloballydispersedteam,itisoneofthemostproductiveprojectseverdocumentedatarunrateoffivetimesindustryaverage.
CONCLUSIONSThiscasestudyisaproofpointfortheargumentthatdistributedteamsandevenoutsourcedteamscanbeasproductive as a small collocated team. This requires excellent implementation of Scrum along with goodengineeringpractices.Theentiresetofteamsmustfunctionasasingleteamwithoneglobalbuildrepository,onetrackingandreportingtool,anddailymeetingsacrossgeographies.
Outsourced teamsmustbehighlyskilledAgile teamsandproject implementationmustenforcegeographictransparencywithcross-functionalteamsatremotesitesfullyintegratedwithcross-functionalteamsattheprimarysite.IntheSirsiDynixcase,theteamswereallrunfromacentralsitegivingstrongcentralcontrol.
ItishighlyunlikelythatdistributedoutsourcedteamsusingcurrentAgileAlliancebestpracticesofdistributingworktoindependentScrumteamsacrossgeographiescouldachievethelevelofperformanceachievedinthiscasestudy.Therefore,SirsiDynixsetsanewstandardofbestpracticesfordistributedandoutsourcedteamswithapreviouslydemonstratedhighlevelofAgilecompetence.
ãJeffSutherland1993-2021 111
FULLYDISTRIBUTEDSCRUM:REPLICATINGLOCALPRODUCTIVITYANDQUALITYWITHOFFSHORETEAMS
JeffSutherland,Ph.DGuidoSchoonheimMauritsRijk
Scrum,Inc.Xebiab.v.Xebiab.vBoston,MA,USHilversum,NetherlandsHilversum,Netherlands
[email protected]@[email protected]
Abstract
Scrumwasdesigned forhyperproductive teamswhereproductivity increasesby5-10 times over industryaveragesandmanycolocatedteamshaveachievedthiseffect.Thequestionforthispaperiswhetherdistributed,offshored teamscanconsistentlyachieve the same levelofperformance. Inparticular, cana teamestablishalocalizedvelocityandqualityandthenmaintainorincreasethatvelocityandqualitywhendistributingteamsacrosscontinents.Since2006,Xebia(Netherlands)startedlocalizedprojectswithhalfDutchandhalfIndianteammembers.Afterestablishinglocalizedhyperproductivity,theymovetheIndianmembersoftheteamtoIndiaandshow the same velocity with fully distributed teams. After running XP engineering practices inside manydistributed Scrum projects, Xebia has systematically productized a model for high performance, distributed,offshoreteamswithoneofthelowestdefectratesintheindustry.
INTRODUCTIONTheadvantagesofcolocatedteams,doublingdeveloperproductivitycomparedtonon-colocatedteams,
arecommonlyeliminatedbydistributedsoftwaredevelopmentandoffshoring [1].Thispaper introducesamodelforproducingdistributedandoffshoredteamvelocitythatisequaltocolocatedvelocityofasingleteam.Themodelisrepeatable,provenacrossmanyprojects,andisrecommendedforteamsthatcanexecuteahighperformanceScrumimplementation[2]withXPengineeringpracticesinside[3].
Agile project management with Scrum derives from best business practices in companies like Fuji-Xerox, Honda, Canon, and Toyota [4]. Combining Scrum with XP engineering practices has generated hyperproductive teams with 5-10 times industry average performance since 1993 [5][6]. In 2005, two Agile companies, SirsiDynix (U.S.) and Exigen Services (Russia), useddistributedScrum teams todeliver linearly scalableperformance fora largeprojectofover1Mlinesofcode[7]. Adistributedteamofover50people in theU.S.andRussiadeliveredvelocityperdeveloperequivalenttoa6personhyperproductive,colocatedScrumteam.This50personteamproducedthesamenumberoffeaturesasatypical350personwaterfallteam[8].
During 2006-2008, Xebia implemented a distributed software development team model on multiple projects of variable types with teams half in the Netherlands and half in India. These distributed teams used the Scrum process with XP engineering practices inside. When replicated over multiple projects, the Xebia implementation shows distributed velocity equal to local velocity and has created a new recommended standard for deploying distributed teams across geographies.
ãJeffSutherland1993-2010 112
Here we describe offshoring strategies for overcoming the typical geographic, language, and cultural barriers that impede distributed development. Traditional outsourcing failures can be avoided with the approach described here. Distribution of individual Scrum teams across geographies eliminates communication failures, XP practices solve integration problems, and daily team meetings maintain high focus on customer priorities. Earlier work in other companies showed that colocation doubled Agile team productivity [1]. The fully distributed model supports geographically transparent software development projects where performance consistently meets or exceeds productivity of colocated Agile teams.
BENEFITSANDCHALLENGESINOUTSOURCINGOFFSHOREU.S.,European,orJapanesecompaniesoftenoutsourcesoftwaredevelopmenttoEasternEurope,Russia,
ortheFarEast.Thethreekeyadvantagesthatoffshoringstrivestoachieveare(1)lowercostsoflabor,(2)capturetalentnotavailablelocally,and(3)increaseanddecreaseprojectsizewithoutlayoffs.Eachoftheseadvantagescomeswithitsownchallengesthathavetobesolvedtomakeoutsourcingsuccessful.
LOWERCOSTOFLABOR
Typically, remote teamsoperate independentlyandcommunicationproblems lowerproductivity.Mostoffshoring organizations require detailed specifications before they begin a project and theses traditionalprojectplanningmethodologiesshowhighfailurerates.
Thehiddencostsofoffshoringaresignificant,beginningwithstartupcosts.Barthelemy[65]surveyed50companiesandfoundthat14%ofoutsourcingtooffshoreoperationswerefailures.Intheremainder,costsoftransitioningtoanewvendoroftencanceledoutanticipatedsavingsfromlowlaborcosts.Theaveragetimefromevaluatingoffshoringtobeginningofvendorperformancewas18monthsforsmallprojects.Asaresult,theMITSloanManagementReviewadvisesreadersnottooutsourcecriticalITfunctionsoffshore.
Secondly,highproductivity isnoteasilyachievable. IDXSystems (nowGEHealthcare)averaged240%productivityimprovementwithScrumduring1996-2000basedonanalysisbyexternalfunctionpointexperts.Outsourcingdevelopmenttoanoffshoredwaterfallteamtypicallysaves20%overinternalwaterfallcosts.AtIDX, it would cost twice as much to get a project done offshore compared to internal Scrum teams. AtPatientKeeper(aMITstartupcompanyin2000)during2004-2007,thebreakevenpointforoutsourcingwasachievedonlywhenIndiandeveloperscostlessthan10%ofAmericandevelopers.BecauseIndianwaterfalldeveloperscost30%ofBostonScrumdevelopers,itcostthreetimesasmuchtogetsoftwaredevelopedbyanIndianwaterfall team.ThePatientKeeperBoardpermanentlyterminatedoutsourcingafterreviewingtheseROIdata.
CAPTURETALENT
Capturingexternaltalentmayalsobeaproblem.JackBlount,CEOofDynixandformerCOOofBorland[19][19][19][19][18]decidednottooutsourcetoIndiaandChinaafterheverifiedthatannualturnoverrateswere30-50%[9].Rathidescribeshowemployee turnover is largelydeterminedby twovariables:person-
ãJeffSutherland1993-2010 113
cultureandperson-jobfit[10].Lackofthesefitsresultsinlowerjobsatisfactionandconsequentlyinemployeeturnover.
SCALEPROJECTSIZEWITHOUTLAYOFFS
Scalingwithremotecapacitygivesyoualocalteamthatremainsstablewhenyouscaledown.Howeverifcoredevelopmentismovedoffshore,knowledgegetslostwhenyoudownsize,causingsevereproblemsandsometimesvendorlock-in.
DISTRIBUTEDSCRUMTEAMMODELSAchievingpromisedbenefitsofoutsourcingrequiresrealcostsavings,stableoffshoreteams,andastrategy
forretainingcoreknowledgeonshore.ThiscanbeachievedwithstableoffshoreAgileteamsthatcanmaintainthesamevelocityasonshoreteamsandwithonshoreteamsthatmaintainthesameknowledgelevelasoffshoreteams.
HereweconsiderthreedistributedScrummodelscommonlyobservedinpractice.
IsolatedScrums-Teamsareisolatedacrossgeographies.
DistributedScrumofScrums–ScrumteamsareisolatedacrossgeographiesandintegratedbyaScrumofScrumsthatmeetsregularlyacrossgeographies.
Fully distributed Scrums – Scrum teams are cross-functional with members distributed acrossgeographies.Thismeansthatasingleteamwillhavemembersinmultiplelocations.AsingleteammighthaveaScrumMasterandtwodevelopersintheNetherlandswhileatesterandfourdevelopersresideinIndia.Theseteammembersshareasinglesprintbacklogandsharecodeownership.IntheSirsiDynixcase,theScrumofScrumswaslocalizedwithallScrumMastersinUtah.AtXebia,ScrumMastersmaybeintheNetherlandsorIndiadependingonprojectneeds.
ãJeffSutherland1993-2010 114
Figure 1: Distributed Scrum Team Strategies
Most offshore development efforts use a degenerative form of the Isolated Scrums model where outsourced teams are not cross-functional and not Agile. Requirements may be created in the U.S. and developed in Dubai, or development may occur in Germany and quality assurance in India. Typically, cross-cultural communication problems are compounded by differences in work style in the primary organization vs. the offshored group. In the worst case, teams outsourced this way are not using Scrum and their productivity is typical of waterfall projects further delayed by cross-continent communications lag time. Implementations of Scrum in a data rich CMMI Level 5 company simultaneously running waterfall, incremental, and iterative projects, showed productivity of Scrum teams was at least double that of waterfall teams, even with CMMI Level 5 reporting overhead [11]. Outsourced teams not using Scrum will in the best case achieve less than half the velocity of a onshore site using Scrum assuming equal talent across teams.
The latest thinking in the Project Management Institute Guide to the Project Management Body of Knowledge (PMBOK) models is a degenerative case of isolated non-Scrum teams . This is a spiral waterfall methodology which layers the Unified Modeling Language (UML) and the Rational Unified Process (RUP) onto teams which are not cross-functional [12]. It partitions work across teams, creates teams with silos of expertise, and incorporates a phased approach laden with artifacts that violate the principles of lean development [13].
Best practice recommended is a Distributed Scrum of Scrums model. This model partitions work across cross-functional, isolated Scrum teams while eliminating most dependencies between teams. Scrum teams are linked by a Scrum-of-Scrums where Scrum Masters (team leaders/project managers) meet regularly across locations. This encourages communication, cooperation, and cross-fertilization and may be appropriate for newcomers to Agile development or those who have offshore limitations that cripple the productivity of the fully distributed model.
ONETEAMMODELXebia’sFullyDistributedScrummodelhasallteamsfullydistributedandeachteamhasmembersatmultiplelocations.Whilethis“OneTeam”modelmightseemtocreatecommunicationandcoordinationburdens,mostcommunicationishandledbyfollowingtheScrumcycle.ThedailyScrummeetingsactuallyhelptobreakdownculturalbarriersanddisparitiesinworkstyleswhilesimultaneouslyenhancingcustomerfocusandoffshoreunderstandingof customerneeds.Onenterprise implementations, it canorganize theproject into a singlewholewithanintegratedglobalcodebase.ProperimplementationofOneTeamprovideslocationtransparencyandperformancecharacteristicssimilartohyperproductiveco-locatedteams.
Isolated Scrums Teams
Distributed Scrum of Scrums
Fully Distributed Scrums
ãJeffSutherland1993-2010 115
MaximumbusinessvalueisdeliveredinScrumbyimplementingtheProductBackloginorderofbusinessvalueoffeatures.Xebiateamproductfeaturesarerepresentedinuserstoriesandsizeofastoryisrepresentedinstorypoints[5].
Xebia teamsconsistentlyvalidate thatdistributedvelocityequals colocatedvelocitybymeasuring costperstorypoint.Thevalueofthefeaturedividedbyactualcostistheprimeindicatorofbusinessvalueandthisisdirectlyproportional to thevelocityof theteaminstorypointsper iteration.TheFullyDistributedScrumsmodelisrecommendedforexperiencedAgileteamsinmultiplelocationsbecausecostperstorypointisthesameaslocalizedteamsandtheXebiadistributedteamshaveimprovedfocusonexecutingstoriesthatbetterfitcustomerneedsintherightorderthanlocalizedteams.
Cost per story point cannot be standardized across the industry. The best standard metric to compareproductivityacrossprojectsisFunctionPoints.CapersJonesdemonstratedmanyyearsagothatthenumberoffeaturesdeliveredinFunctionPointscanbeestimatedby“back-firing”usinglinesofcodedelivered[8].Whilethis is an indirectmeasure of business value delivered, it is the bestmeasure available to compare teamsindustrywide.
Whileonemightarguethatdeliveringlotsofcodemaynotproducebusinessvalue,thisiscontrolledbyScrumteamsrunningXPengineeringpracticesintwoways:
• ScrumordersProductBacklogbybusinessvalueandassureslinesofcodedelivereddirectlyincreasebusinessvalue.
• TheXPpracticeofrefactoringeliminatesmanythousandsoflinesofcodethatwouldremainstaticinthecodebaseofawaterfallteam.
ThenetresultisthatcomparisonsofbusinessvaluedeliveredbyScrum/XPteamsisconservativecomparedtowaterfallteamswhenmeasuredbyanyindicatoraffectedbylinesofcode.
ThusthemessageofthispaperisthatXebiaScrum/XPteamsdeliverFunctionPointsoverseventimesfasterthanindustryaveragewaterfallteamsandtheFunctionPointstheydeliverhavehigherbusinessvaluethanthewaterfallteamsbyoveranorderofmagnitude.Sincethisvalueisdeliveredatthesamecostperstorypoint,and this cost is a direct indicator of business value, either locally or distributed, the OneTeam model isrecommendedfordistributeddevelopmentbythoseAgileteamscapableofexecutingit.
XEBIAPRORAILPUBCASESTUDYThemodelforFullyDistributedScrumsisbestillustratedbyareallifeexampleofaXebiaOneTeamproject;theProRailPUBproject.
ãJeffSutherland1993-2010 116
ProRail,thelogisticalandinfrastructuralpartoftheDutchrailways,hasbeendevelopinganewinformationsystemfortravelers.Informationabouttraindeparturetimesisstoredcentrallyandupdatedwithinformationfromtherailnetwork.Whenatrainisdelayedorarrivesearlythisinformationiscapturedbysensorsintheinfrastructureaswellasbymanualactionstoupdatetraininformation.
ThepublishingofthisinformationtotravelersonalltherailwaystationsthroughouttheNetherlandsisthescope of Xebia’s development assignment. Development included the aggregation and distribution system(combiningrealtimeinformationaboutmultipletrainsintomessagesrelevantforstations),theclientinthedisplays, theaudiosystemandthecontrollingandmonitoring interfaces.As this isamissioncritical,high-availabilityenterprisesystemwithlargevisibility,thenon-functionalrequirementsareextensive.
Xebia tookover thisproject froma failedwaterfall implementationandmeetingdeadlineswasnowakeycriteria.ThetransparencyandempiricalprojectcontrolthatScrumdeliverswerekeyincentivesfortheclienttoengageXebia.Thechoicetomakeitanoffshoreprojectwasdrivenbycostandscalability.
WhileScrumissimpletounderstand,itisnoteasytoimplementanddistributeddevelopmentaddsanotherlayerofcomplexity.ThePUBprojectencounteredanumberofchallengesintheseareas.
LOWERCOSTOFLABORTheOneTeamapproachforFullyDistributedScrumteamsdeliversthesameresultsasawellrunningcolocatedScrumteaminanoffshoringsituation.DifferentaspectsofthePUBprojectcanillustratethis.
PRODUCTIVITY.VelocityofaScrumteam isdeterminedby thenumberof storypoints that the teamcancompleteusingastandardizeddefinitionof“done”inasingleiteration.AsstorypointsarenottranslatablebetweenprojectsthePUBprojectsizehasalsobeenmeasuredinfunctionpoints.Thismeasurehasbeendoneforboththeold(failed)implementationandthenewimplementationbyXebiaandthesefigurescorrespond.Asameasurementthisdoesnotgiveacompletelyaccuratepicture,asitdoesnotcapturetheamountofbusinessvaluedeliveredverywell.Itishoweverthebestmeansavailabletomakecomparisonsoverprojects.Belowisatabletakenfromacolocated6personScrum,theSirsiDynixfullydistributedScrumproject,andextendedwithPUBdata.
ãJeffSutherland1993-2010 117
Cohn
ColocatedScrum
Cohn
Waterfall
SirsiDynixDistributedScrum
XebiaDistributedScrum
PersonMonths
54 540 827 125
LinesofJava 51000 58000 671688 100000+
FunctionPoints
959 900 12673 1887
FP per dev.permonth
17.8 1.7 15.3 15.1
Table1:ProductivityofColocatedScrumvs.WaterfallTeam[5],SirsiDynixDistributedScrum[9],andXebiaOneTeam.
AswecanseetheScrumprojectseasilyoutperformtheWaterfallproject.XebiaDistributedScrumcomesclosetothesmallcolocatedScrumteam.TheperformanceoftheSirsiDynixandXebiaprojectisverysimilar.ThisshowsthatthehighperformancefullydistributedScrumapproachisreproducibleandnottypicalforonlytheSirsiDynixenvironment.
To investigate the effect of distributing teams on the productivitywe can look at the cost per story pointdeliveredthroughouttheproject.
Figure3:Hoursperstorypointduringtheproject
Itisimportanttonotethegradualincreaseinstorypointcostduringthelifeofmostprojectsduetogrowingcomplexityandgrowingcodebase.Thisconstanthasbeencompensatedfortofocustheabovediagramonanyoutliers.Thetransitionfromalocalteamtoadistributedteamtookplaceatiteration6.Ascanbeseenfromthe resulting graph, the number of hours needed to implement a story point was not affected by thisdistribution. Storypoint estimateswere determined at the beginning of the project for thewhole productbacklogandweredeterminedfornewrequirementsastheysurfaced.Iteration18and19showasignificantincrease inhoursneededper storypoint.Technicaldebthadbeenbuiltupduring theprevious iterations.Startingwith iteration 20 this technical debtwas consistently removed, resulting in a gradual increase inproductivity.
Hours/Storypoint
02468101214161820
1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627
Iteration
Hours
ãJeffSutherland1993-2010 118
HIGHQUALITYANDCONSISTENCY.ThroughoutthecourseofthePUBprojectalotofattentionhasbeenpaidtoquality.TheScrumdefinitionofdone for this project includes unit test coverage of at least 80%, fully automated functional testing, fullregressiontesting,performanceandloadtestingforallimplementedstoriesaswellasupdatingthenecessarydocumentation.
Foreverypieceoffunctionalitythewholeteamdiscussesproperdesignandnecessaryrefactoringtakesplace.In addition to this sharedownershipoverdesign every teamemploys a ‘qualitywatchdog’. This is a teammemberaccountableforqualityandconsistency.Anyproblemsthathe/shesignalsaretobepickedupanddiscussed by the team. All teams share the same team room and team members participate in designdiscussionsofotherteamsinordertomaintainarchitecturalconsistencyacrossteams.Pairprogrammingandrotationofpeoplebetweenteamsisusedtoavoidcodeownershipandspreadknowledge.
Allissuesthatarefoundoutsidetheiterationaremeasuredandsolvedasshowninthefollowinggraph.
Figure4:Opendefectsduringtheproect
Cumulative vs. open defects
0100200300400500600700800900
1 3 5 7 9 11 13 15 17 19 21 23 25 27Iteration
ãJeffSutherland1993-2010 119
The above figure shows that thenumbef opendefects remains constant (around50). Thismeans that theprojectisnotbuildinguptechnicaldebtduringdevelopment.ThenumberofopenbugsperKLOCisactuallydecreasingbecausethecodebaseiscontinuouslygrowing.Otherdataalsoshowsthatmorethan90%ofthedefectsfoundaresolvedinthesameiterationinwhichtheywereintroduced.
Basedonthesenumberswecanconcludethattheverificationandvalidationprocesshasisolated6defectsperKLOC.Duringacceptancetestslessthan1defectperKLOCwasfound.Afairestimateisthat50%ofthedefectsarestill leftintheproductaftertheacceptancetest,leavinguswith1defectperKLOC.Thisisfarlessthanindustryaverage,whichisaround5defectsperKLOC[14].FullyDistributedOneTeamScrumsapplyingXPpracticesproduceextremelyhighquality.
CAPTURETALENTNOTAVAILABLELOCALLYAsdescribedin2.2thisbenefitdependsontheemployeeturnover.Tocopewithhighturnoverratestheprojectconcentratedonclearcommunication,andspecialattentiontothepeople-cultureandpeople-jobfit.InadditiontheAgilewayofworkingprovidestalentedpeoplewithresponsiblework,thusguaranteeingjobsatisfaction.Thisresultedinaturnoveroflessthan5%inoneyear.
CULTURALDIFFERENCES.Indian and Dutch team members have a different background and culture. This shows most clearly incommunication.Forexample,whereDutchteammemberscanbeloudanddirect,Indianteammemberscanbecarefulandcautiousintheirexpression.AlsoIndiaismorehierarchicallyorientedthantheNetherlands.Thefirstandmostimportantthingtocounterthesedifferencesisgoodpersonalrelationships.Bytravelingatthebeginningandthroughouttheproject,byseeingeachotherdailyinvideoconferencestand-ups,andbybeingpartofthesameteampersonalrelationshipsformed.Secondly,ateamcultureaimedatopennessanddirect communicationwas actively developed by the ScrumMasters. This helped bring out issues duringretrospectivesand loweredcommunicationbarriers.Thirdly,acompanycultureofopennesswithanequalvaluesystemonbothsitessupportedtheteamcultureandmadeidentifyingwitheachothereasier.
SHARINGCONTEXTANDPRIORITIESInanoffshoringsituationitisdifficulttofullycommunicateallclientnuances,contextandprioritiestooffsiteteammembers.ToactivelydistributethisknowledgeXebiascheduledregulartraveling,always-openSkypeconnections,aprojectnewsgazetteaftereveryiterationandinformalupdatesbytheproductowner.
CLEARCOMMUNICATIONTHROUGHSCRUMTheScrummeetingsfacilitatepracticallyallnecessarycommunication.Thisisonlypossiblebecausetheteamisfullydistributedandsharesthesamesprintgoals.AllScrummeetingsweredoneinadistributedwayusingvideoconferencingviaasimpleSkypevideocallwiththeexceptionoftheDemo.SeparatemeetingroomsaresetupwithconferenceequipmentandaScrumplanningtoolwithadigitalburndownchartisusedtoshare
ãJeffSutherland1993-2010 120
thestatusof thesprintacross locations.Amicrophone ispassedaroundas ‘talkingstick’ to facilitateclearaudibility. Xebia found that face to face visuals greatly increases the effectiveness of communication andenhancespersonalrelationships.
TheSprintplanningmeetingisdonewiththewholeteamusingplanningpoker[15]sothatmembersonbothshorescontributetotheestimationprocess.Planningadistributedsprinttook4hoursonaverageusingtwoweeksprints.
ThedailystandupmeetingsaredonewhenthedevelopersintheNetherlandscometowork.Adistributedstandup lastsno longerthen15minutes.AScrumofScrumsmeetingwasheldbyScrumMastersafter thestand-upstosynchronizeanydependantissuesorimpedimentsaswellastechnologicalissues.
The Scrum demo was not distributed in this case to provide maximum focus and responsiveness to thecustomer.TheDutchmembersbriefedtheIndianmembersaftereveryDemo.TheScrumretrospectivegoesinthesamefashionastheSprintplanningmeeting.Thedistributedretrospectiveiscompletedin2hours.
Together these meetings provide the full official meeting cycle. One on one meetings are being held asnecessary,aswellasdesigndiscussions.ThisisnodifferentfromacolocatedScrumwiththepossibleexceptionoftooling.
SOMEWORKISLOCALWhilealldevelopmentworkcanbedistributedthereisprojectworkthatisnoteasilydoneinadistributedway.AfourthScrumteam,consistingonlyoflocalteammembers,wasdedicatedtospecificcustomerfacingcompliancy activities and removing certain impediments. Examples of local deliverables arewritingDutchdocumentation, aligningwith customer architectural stakeholders, discussing requirementswith technicalstakeholders and researching technical dependencies between the infrastructure and other systems. Thisresultedinclearingofalotofroadblocksandahighvelocityforthedistributedteams.
TOOLINGFORCOMMUNICATIONANDPROCESSInthisprojectScrumWorks[16]wasusedtomanagetheproductbacklogandsprintbacklogelectronically.Burndowngraphswereprintedeverydayandpostedonthewallintheteamrooms.
Forglobalsharingofinformationanddocumentationawikiwasusedintensively.Todiscussarchitectureasmartboard (computerizedwhiteboard)was used, alongwith other solutions for digital whiteboarding. Asinglecoderepository,singlecontinuousbuildsystem,testserversaccessiblefrombothlocationsandasharedmailinglistaresomeofthetoolsusedtofacilitatethedevelopmentprocess.
ãJeffSutherland1993-2010 121
PROJECTSTRUCTUREANDSCALINGXebiainitiatedthePUBprojectwithashortinitiationphasewheretheproductbacklogwasdeveloped,basicarchitecture constraints were established and processes such as QA, Acceptance and RequirementsmanagementweresetupwiththecustomertomatchtheclientorganizationinanAgileway.
Afterthreeweeksofprojectinitiation,acolocateddevelopmentteamstartedthefirstiterationoftheproject.Iterationlengthwassetattwoweeksthroughouttheproject.
ThefirsttwoiterationsweredonewithDutchdevelopers.Indianteammemberswereincludedonsiteassoonasimmigrationandlogisticalconstraintsallowed,startingwiththethirditeration.BothDutchandIndianteammembersworkedasasinglecolocatedScrumteamwithasinglesprintbacklog,followingallXPengineeringpractices.Inthesharedonsiteiterationstheteammembersforgedpersonalrelationshipstolastthroughouttheproject.Bybeingonsitewiththecustomer,theIndianteammembersacquiredagoodsenseofcustomercontext. Italsogoteveryonealignedconcerningpractices,standards, tooling,andnaturalroles in theteamformed. After three iterations the onsite Indian team members returned to India. During these 5 initialiterations(10weeks)theteamestablishedcolocatedhyperproductivity.
TheprojectscaledupafterIndianteammembersreturnedhome.Engineerswereaddedandtwonewfullteamswereformed.EngineersintheNetherlandsweresplitoverbothteamsaswereengineersinIndia,creatingtwoteams thatbothhavemembers inmultiple locations.Careful attention ispaid to spreading the experienceamongthenewteamsandpracticeslikepairprogrammingareusedtogetnewmembersuptospeed.Thiscelldivisionlikeprocessisrepeateduntiltheprojectisatthedesiredscale.
Figure 2: Fully Distributed Scrum team division
TheprojectscaleduptothreefullydistributedScrumteamsandafourthlocalScrumteam,withatotalof25people.Thedifferentteamssharedthesameproductbacklogbutusedtheirownsprintbacklogs.
ãJeffSutherland1993-2010 122
Attheendoftheprojecttheteamswerescaleddownandmerged.AstheclientpreferredtoworkwithDutchengineers formaintenance the Indiansidewasscaleddown further.Thiswasnoproblemsince theuseofdistributedteamsalsoensuresdistributedknowledge.
ThetotalsizeoftheXebiarealizationonthisprojectisabout20man-years,100.000+linesofcodeoveraperiodof11months.
CONCLUSIONSInsummary,itispossibletocreateadistributed/outsourcedScrumwiththesamevelocityandqualityasacolocatedteamandthiscapabilityhasbeenreproducibleovermanyprojects.TheOneTeamstrategylowercosts,capturesoffshoretalentandallows increasinganddecreasingteamsizewithoutknowledge loss.WehighlyrecommendthisstrategyforexperiencedAgileteams.
FUTURERESEARCHSeveralotherXebiaprojectsachievedthesamevelocityandqualityasthePUBprojectconfirmingtheOneTeamcapabilityofdistributinglocalizedvelocityandqualityacrosscontinents.However,duringthePUBprojectdatacollectionwas standardized and refined to a high level. Future projectswill use the same data collectionstandardsasthePUBprojectallowinga largerprospectivestudyoftheOneTeameffectwithcomparabilityacrossmanyprojects.
WhileXebiahasnowrepeatedlydemonstratedtheeffectivenessofthefullydistributedteammodel,itrequiresAgileteamstofullyimplementthepracticesofScrumandXP.Lessthan10%ofAgileteamsworldwidearecapableofdoingthisin2008.AdditionalstudiesofthefullydistributedmodelacrossmultiplecompanieswouldbeusefulandmayhelpsomecompaniesmovebeyondpartialimplementationofAgilepracticesinordertoachievethefullydistributedmodelbenefits.
REFERENCES[1] S.Teasley,L.Covi,M.S.Krishnan,andJ.S.Olson,"HowDoesRadicalCollocationHelpaTeamSucceed?,"
inCSCW'00Philadelphia,PA:ACM,2000,pp.339-346.
[2] J. Sutherland, "Future of Scrum:Parallel Pipelining of Sprints in ComplexProjects," inAGILE2005ConferenceDenver,CO:IEEE,2005.
[3] K.Beck,ExtremeProgrammingExplained:EmbraceChange.Boston:Addison-Wesley,1999.
[4] H.Takeuchi and I.Nonaka, "TheNewNewProductDevelopmentGame,"HarvardBusinessReview,1986.
[5] M.Cohn,UserStoriesApplied:ForAgileSoftwareDevelopment:Addison-Wesley,2004.
[6] J.SutherlandandK.Schwaber,TheScrumPapers:Nuts,Bolts,andOriginsofanAgileMethod.Boston:Scrum,Inc.,2007.
[7] J. Sutherland, A. Viktorov, and J. Blount, "Adaptive Engineering of Large Software Projects withDistributed/Outsourced Teams," in International Conference on Complex Systems Boston,MA, USA,
ãJeffSutherland1993-2010 123
2006.
[8] C.Jones,Softwareassessments,benchmarks,andbestpractices.Boston,Mass.:AddisonWesley,2000.
[9] J.Sutherland,A.Viktorov,J.Blount,andN.Puntikov,"DistributedScrum:AgileProjectManagementwith Outsourced Development Teams," in HICSS'40, Hawaii International Conference on SoftwareSystemsBigIsland,Hawaii:IEEE,2007.
[10] N.Rathi, "Humanresourcechallenges in Indiansoftware industry:Anempiricalstudyofemployeeturnover,"Mercer,2004.
[11] J. Sutherland, C. Jacobson, and K. Johnson, "Scrum and CMMI Level 5: A Magic Potion for CodeWarriors!,"inAgile2007,Washington,D.C.,2007.
[12] K.E.NidifferandD.Dolan,"EvolvingDistributedProjectManagement,"IEEESoftware,vol.22,pp.63-72,Sep/Oct2005.
[13] M. Poppendieck, "A History of Lean: From Manufacturing to Software Development," in JAOOConference,Aarhus,Denmark,2005.
[14] W.S.Humphrey,IntroductiontothePersonalSoftwareProcess:AddisonWesley,1996.
[15] M.Cohn,AgileEstimationandPlanning:Addison-Wesley,2005.
[16] Danube,"ScrumWorksPro."vol.2008Seattle:Danube,2008.
ãJeffSutherland1993-2010 124
SCRUMANDCMMILEVEL5:THEMAGICPOTIONFORCODEWARRIORS
JeffSutherland,Ph.D.
PatientkeeperInc.
CarstenRusengJakobsen
SystematicSoftwareEngineering
KentJohnson
ABSTRACTProjectscombiningagilemethodswithCMMI2aremoresuccessfulinproducinghigherqualitysoftwarethatmoreeffectivelymeetscustomerneedsatafasterpace.SystematicSoftwareEngineeringworksatCMMIlevel5andusesLeanproductdevelopmentasadriverforoptimizingsoftwareprocesses.ValuableexperiencehasbeengainedbycombiningAgilepracticesfromScrumwithCMMI.
EarlypilotprojectsatSystematicshowedproductivityonScrumteamsalmosttwicethatoftraditionalteams.Otherprojectsdemonstratedastorybasedtestdrivenapproachtosoftwaredevelopmentreduceddefectsfoundduringfinaltestby40%.
WeassertthatScrumandCMMItogetherbringamorepowerfulcombinationofadaptabilityandpredictabilitytothemarketplacethaneitheronealoneandsuggesthowothercompaniescancombinethem.
INTRODUCTIONOneofthetrendsinthesoftwareindustryissoftwareprojectsaremorecomprehensiveandcomplexwhilecustomersatthesametimerequestfasterdeliveryandmoreflexibility.Successfulsoftwaredevelopmentischallengedbythesupplier’sabilitytomanagecomplexity,technologyinnovation,andrequirementschange.Customerscontinuallyrequestssolutionsfaster,betterandmorecost-effective.AgileandCMMImethodsbothaddressthesechallengesbuthaveverydifferentapproachandperspectiveinmethodsapplied.
Managementofcomplexityrequiresprocessdiscipline,andmanagementofincreasedspeedofchangerequiresadaptability.CMMIprimarilyprovidesprocessdisciplineandScrumenhancesadaptability.Thisleadstothequestion,whetherornotitispossibletointegrateCMMIandagilepracticeslikeScrumtoachievethebenefitsfromboth–orevenmore?
2®CapabilityMaturityModel,CMMandCMMIareregisteredintheU.S.PatentandTrademarkOffice
ãJeffSutherland1993-2010 125
Thispaperprovidesananalysisoftheeffectof introducingAgilepracticeslikeScrumandstorybasedtestdrivensoftwaredevelopmentandknowledgegainedonwhatisrequiredtobeCMMIcompliant,whilerunninganAgilecompany.
CMMITheCapabilityMaturityModel(CMM)hasexistedsince1991,asamodelbasedonbestpracticesforsoftwaredevelopment.Itdescribesanevolutionarymethodforimprovinganorganizationfromonethatisadhocandimmaturetoonethatisdisciplinedandmature[71].TheCMMisinternationallyrecognizedandwasdevelopedbytheSoftwareEngineeringInstituteatCarnegieMellonUniversity,Pittsburgh,USA.
In 2002, a new and significantly extended version called CMMI was announced, where the ‘I’ stands for‘Integration’[72].Thismodelintegratessoftwareengineering,systemsengineeringdisciplines,andsoftwareacquisitionpracticesintoonematuritymodel.CMMIdefines25processareastoimplement.Foreachprocessarearequiredgoals,expectedpracticesandrecommendedsub-practicesaredefined.Inadditionasetofgenericpracticesmustbeappliedforallprocesses.
Thepast15yearsofexperiencewithCMMandCMMI,demonstratesthatorganizationsappraisedtohigherlevelsofCMMorCMMIimprovetheabilitytodeliveronschedule,cost,andagreedquality.Increasingly,theindustryrequiressupplierstobeappraisedtoCMMorCMMIlevel3orhigher[73].Anumberofgovernmentalorganizations worldwide, have established CMMImaturity requirements. Recently the Danish Minister ofScience proposed regulations to require public organizations to request documentation of their supplier’smaturity[74].
SCRUMScrum for softwaredevelopment teamsbeganatEaselCorporation in1993 [22] andemergedas a formalmethod at OOPSLA’95 [23]. A development process was needed to support enterprise teams wherevisualization of design immediately generatedworking code. Fundamental problems inherent in softwaredevelopmentinfluencedtheintroductionofScrum:
• Uncertaintyisinherentandinevitableinsoftwaredevelopmentprocessesandproducts-Ziv’sUncertaintyPrinciple[54]
• Foranewsoftwaresystemtherequirementswillnotbecompletelyknownuntilaftertheusershaveusedit-Humphrey’sRequirementsUncertaintyPrinciple[58]
• Itisnotpossibletocompletelyspecifyaninteractivesystem–Wegner’sLemma[55]
• Ambiguous and changing requirements, combined with evolving tools and technologies makeimplementationstrategiesunpredictable.
ãJeffSutherland1993-2010 126
“All-at-Once”modelsofsoftwaredevelopmentuniquely fitobject-oriented implementationofsoftwareandhelp resolve these challenges. They assume the creation of software involves simultaneous work onrequirements,analysis,design,coding,andtesting,thendeliveringtheentiresystemallatonce[32].
SutherlandandSchwaber,co-creatorsofScrumjoinedforceswithcreatorsofotherAgileprocessesin2001towrite the Agile Manifesto [75]. A common focus on working software, team interactions, customercollaboration,andadaptingtochangewereagreeduponascentralprinciplesessentialtooptimizingsoftwareproductivityandquality.
CMMIANDAGILEMETHODSSoonafterpublicationoftheAgileManifestoin2001,MarkPaulkprincipalcontributorandeditorofCapabilityMaturityModelVersion1.0[71],observedthatAgilepracticesareintendedtomaximizethebenefitsofgoodpractice [76, 77]. “The SW-CMM tells what to do in general terms, but does not say how to do it; agilemethodologiesprovideasetofbestpracticesthatcontainfairlyspecifichow-toinformation–animplementationmodel–foraparticularkindofenvironment.”However,Paulknotedthataligningtheimplementationofagilemethodswiththeinterestsofthecustomerandotherstakeholdersinagovernmentcontractingenvironmentforsoftwareacquisitionmightbeanimpossibletask,wherehighcustomerinteractionisdifficult.
Surdu[78]andMcMahon[79]reportedpositiveexperiences in2006usingagileprocessesongovernmentcontractswhilenotingtheneedforprocessdiscipline,goodsystemengineeringpractices,anddevelopmentofself-motivated teams.Collaborationwith customerswasachieved throughagile educationandnegotiation.ThesestudiesprovidepracticalconfirmationofPaulk’sanalysisoftheapplicabilityofagilepracticesinaCMMenvironment.
Paulk[77]pointsoutthat“Whenrationallyimplementedinanappropriateenvironment,agilemethodologiesaddressmanyCMMlevel2andlevel3practices.”SimilarlyKaneandOrnburnpresentamappingofScrumandXPtoCMMI[80]demonstratingthatamajorityoftheCMMIprocessareasrelatedtoProjectManagementcanbeaddressedwithScrumandthemajorityofprocessareasrelatedtosoftwareengineeringcanbeaddressedwithXP.CMMIexpectsthatprocessesareoptimizedandperhapsreplacedovertimeandstates:“Optimizingprocessesthatareagileandinnovativedependsontheparticipationofanempoweredworkforcealignedwiththebusinessvaluesandobjectivesoftheorganization.”[72](page49).
WeagreewiththeauthorsabovethatAgilemethodologiesadvocategoodengineeringpracticesthatcanbeintegratedintheCMMIframework,andconsiderthelargestdrawbackofmostAgilemethodologiesisalimitedconceptofinstitution-widedeployment.InstitutionalizationiskeytoimplementationofallprocessesinCMMI,andisstronglysupportedbyasetofGenericPractices.It isourbeliefthatthesepracticescouldbeusedtoensurethatAgilemethodologiesareinstitutionalizedinanyorganization.
AgilemethodslikeScrumandXParepracticalmethodsthatcansupportdifferentpartsofCMMI.CombiningScrumandCMMIpracticescanproduceamorepowerfulresultthaneitheraloneandcanbedoneinwaywhereCMMIcomplianceismaintained.AmoredetailedanalysisofafullimplementationoftheScrumdevelopment
ãJeffSutherland1993-2010 127
processalongwithsomeXPengineeringpracticesusedatSystematicshowsquantitativeresultsofintroducinggoodagilepracticesandhowtomaintainCMMIcomplianceinanAgilecompany.
SCRUMANDCMMI:AMAGICPOTIONSystematicwasestablishedin1985andemploys371peopleworldwidewithofficesinDenmark,USAandtheUK.ItisanindependentsoftwareandsystemscompanyfocusingoncomplexandcriticalITsolutionswithininformation and communication systems. Often these systems are mission critical with high demands onreliability,safety,accuracyandusability.
Customers are typically professional IT-departments in public institutions and large companies withlongstandingexperienceinacquiringcomplexsoftwareandsystems.SolutionsdevelopedbySystematicareused by tens of thousands of people in the defense, healthcare, manufacturing, and service industries.Systematicwas appraised 11November 2005using the SCAMPISM3method and found to be CMMI level 5compliant.
WorkingatCMMIlevel5bringsmanyadvantages.Systematichasfirsthandexperienceofreductioninreworkby38%to42%overearlierlevels,estimationprecisiondeviationlessthan10%,and92%ofallmilestonesdeliveredearlyorontime.Atthesametime,extraworkonprojectshasbeensignificantlyreduced.
Moreimportantly,Systematichastransformedovertwentyyearsofexperienceintoaunifiedsetofprocessesusedbyallsoftwareprojects.Historicaldataaresystematicallycollectedandanalyzedtocontinuouslyprovideinsightintothecapabilityandperformanceoftheorganization.
Theuseofasharedcommonprocessmakes iteasier forpeopletomovefromproject toprojectandshareexperiencesandlessonslearnedbetweenprojects.Insightintothecapabilityandperformanceofprocessesmakesitpossibletoevaluateperformanceofnewprocessestoperformanceofexistingprocesses.Andthisformsthefoundationforcontinuousimprovement.
3SMCapabilityMaturityModelIntegration,andSCAMPIareservicemarksofCarnegieMellonUniversity
ãJeffSutherland1993-2010 128
FIGURE1:CMMIANDSCRUMPRODUCTIVITYGAINS
Inshort,Systematicisabletodeliverwhatthecustomerhasorderedonschedule,costandqualityusing69%effortcomparedtoaCMMILevel1company[81,82].Thisbenefitcomesattheminimalcostof9%processfocusinprojectmanagementandengineering.CMMILevel5isincreasinglyarequirementfromcustomersandkey toobtaining largecontracts, especiallywithindefenceandhealthcare.Customers recognize thatCMMILevel5giveshighpredictabilityandbetter-engineeredproduct forscalability,maintainability,adaptability,andreliability.
EarlyresultsindicatethatwhenCMMItraditionalprocessesareoptimizedusingScrum,theproductivityforlargeprojectsisdoubledandtheamountofreworkisreducedanadditional40%overthatofCMMILevel5companies. It is important to note that the optimized process is amixed process, using traditional CMMIprocessestoestablishaprojectbaselineexpressedasaproductbacklogcombinedwithScrumasthepreferredwaytoimplementtheprojectiniterationsofonemonthSprints.ThecombinationoftheCMMIandScrumintotheoptimizedCMMIScrumprocessincludestheproperactivitiestoestablishsufficientplanningneededbyboth customer and supplier, andat the same time the flexibility andadaptabilityprovidedby Scrum.ThiscombinedprocessistreatedsimilarlytoanyotherprocessinCMMI.
CMMIprovidesinsightintowhatprocessesareneededtomaintainadisciplinedmatureorganizationcapableof predicting and improving performance of the organization and projects. Scrum provides guidance forefficientmanagementofprojectsinawaythatallowsforhighflexibilityandadaptability.Whenmixingthetwo, a magic potion emerges, where the mindset from Scrum ensures that processes are implementedefficientlywhileembracingchange,andCMMIensuresthatallrelevantprocessesareconsidered.
10%
20%
a
30%
50%
40%
60%
CMMI 1 CMMI 5
70%
80%
90%
100%
CMMI 5SCRUM
CMMI 1Project effort Rework
Work
Proces focusCMMI
SCRUM
50 %
50 %
50 %
10 %
9 %
6 %
25 %
4 %
100 %
69 %
35 %
10%
20%
a
30%
50%
40%
60%
CMMI 1 CMMI 5
70%
80%
90%
100%
CMMI 5SCRUM
CMMI 1Project effort Rework
Work
Proces focusCMMI
SCRUM
50 %
50 %
50 %
10 %
9 %
6 %
25 %
4 %
100 %
69 %
35 %
ãJeffSutherland1993-2010 129
IndividuallyCMMIandScrumhasprovenbenefitsbutalsopitfalls.AnAgilecompanymayimplementScrumcorrectlybutfailduetolackofinstitutionalization,(seesection0)orinconsistentorinsufficientexecutionofengineeringormanagementprocesses.CMMIcanhelpAgilecompaniestoinstitutionalizeAgilemethodsmoreconsistentlyandunderstandwhatprocessestoaddress.
AcompanycancomplywithCMMI,butfailtoreachoptimalperformanceduetoinadequateimplementationofprocesses. Scrum and other Agile methodologies can guide such companies towards more efficientimplementationofCMMIprocessrequirements.
HOWSYSTEMATICADOPTEDSCRUMHerewedescribethegenericstepsoftheprocessSystematicexecutedthatresultedintheadoptionofScrum,earlytesting,andstorybaseddevelopment.
IdentifyBusinessObjectivesandNeeds.CMMIstates [72] (page55) that “successfulprocess-improvementinitiatives must be driven by the business objectives of the organization”. Business objectives and needs areaddressedbythestrategyoftheorganization.
SystematicmadeastrategicdecisiontouseLeanasthedominantparadigmforfutureimprovements.Leanhasdemonstratednotableresultsformanyyearsindomainssuchasautomanufacturing,andduetoit’spopularity,has been adapted to other domains, including product and software development. It was expected thatadoptionofaLeanmindsetwouldfacilitateamoreefficientimplementationofCMMI.
The strategic decision to use Lean as a dominant tool for optimization of processes, is input to CMMIOrganizational Process Focus (OPF) and driven by an organizational shared function for processimprovements.
Analysis.DifferentLeandialectswereevaluatedandLeanSoftwareDevelopment[8]wasidentifiedasthedialectmostrelevanttoSystematic.LeanSoftwareDevelopmentisanagiletoolkit.AcarefulinterpretationoftheAgileManifestoshowsthatthisisnotnecessarilyinconflictwithCMMILevel5.
TheAgileManifestorecognizesthatprocesses,tools,comprehensivedocumentation,contractnegotiationandfollowingaplanhavevalue,butemphasizespeople, interactions,workingsoftware,customercollaborationandrespondingtochangetohavemorevalue.“Theagilemethodologymovementisnotanti-methodology;infact,manyofuswanttorestorecredibilitytotheword.Wealsowanttorestoreabalance:Weembracemodeling,butnotmerelytofilesomediagraminadustycorporaterepository.Weembracedocumentation,butnottowastereamsofpaper innever-maintainedandrarelyusedtomes.Weplan,butrecognizethe limitsofplanning inaturbulentenvironment.”[75]
ãJeffSutherland1993-2010 130
SuccessfulapplicationofLeanSoftwareDevelopment-anagiletoolkit,dependsontheadoptionofanagilemindsettosupplementtheprocessfocus.SystematicvaluesareconsistentwiththeAgileManifesto,andspecialfocuswasplacedonthefollowingaspectsfornewimprovements:
Individuals and interactions. Empowerment: the person executing the process is also responsible forupdatingtheprocess.
Workingsoftwareoverdocumentation.Criticalevaluationofwhatpartsofthedocumentationorprocesscanberemovedorrefinedtoincreasethecustomersperceivedvalueoftheactivitiesisessential.
Respondingtochange.Determininghowtheprocesscouldbeimprovedtosupportreducedcycletimedrovecustomervalue.
Leancompetenceswereestablished,throughhandoutofbooks,formalandinformaltraining,andwalk-the-talkactivities.ProjectManagersweretrainedinLeanSoftwareDevelopment,andMaryPoppendieck[8]visitedSystematicforamanagementseminaronLeanSoftwareDevelopment.
This seminar established an understanding of the Agile and Leanmindset. Based on this training, causaldependenciesbetweentheprinciplesandtoolsinLeanSoftwareDevelopmentwereanalyzed,andasaresulttestpracticesandreducedcycletimewereidentifiedasgoodcandidatesforinitialactivities.TheyrepresentedagoodstartingpointforimplementingLeanandatthesametimefocusedonprocesseswhereimprovementswouldhavesignificanteffectonefficiency.
Pilot.Leanadvocatesthatthepeopleperformingaprocessshouldberesponsibleandempoweredtomaintainthatprocess.IntheintroductiontotheCMMIOPFprocessareaCMMIsaysthesamething.
AnanalysisofthecausaldependenciesinLeanSoftwareDevelopmentledtothedecisiontoseekimprovementsbasedontheprinciplesofAmplifyLearning,DeliverFast,andBuildIntegrityIn.
Selectedprojectswereaskediftheywouldliketopilotimprovedprocessesrelatedtotestandreducedcycletimerespectively.ProjectstaffweretrainedintheLeanmindsetandaskedtosuggesthowtoadoptLeanintotheirprocesses.TheresultwasaselectionofScrumandearlytestingbasedonstory-baseddevelopment.
ãJeffSutherland1993-2010 131
The result of the pilotswere two-fold: it confirmed the general idea of using Leanmindset as source foridentificationofnewimprovements,andsecondlyitprovidedtwospecificsuccessfulimprovementsshowinghowagilemethodscanbeadoptedwhilemaintainingCMMIcompliance.
Implementation. ItwasdecidedtoadoptScrumandstorybasedsoftwaredevelopment in theorganization.ProcessActionTeams(PATs)wereformedtointegratetheexperienceandknowledgegainedfromthepilots,intotheprocessessharedbyallprojectsintheorganization.ThePATswerestaffedwithpeoplethatwouldbeexpectedtoexecutethenewprocesswhenreleased.
Thelargestchangetoprojectplanningisthatfeaturesandworkareplannedinsufficientdetailasopposedtoacompleteinitialdetailedanalysis.TheresultisaScrumProductBacklogwithacompleteprioritizedlistoffeatures/workfortheproject.Allfeatureshaveaqualifiedestimate,establishedwithadocumentedprocessandthroughtheuseofhistoricaldata,but thegranularityof the features increaseas thepriority falls.Theuncertaintythatremainsishandledthroughriskmanagementactivities.
Theprimarychangetoprojectexecutionprocesses,wastointegrateScrumasmethodforcompletingsmalliterations(Sprints),onaselectedsubsetoftheworkwithhighestpriority.
ThisworkverifiedthatScrumcouldbeadoptedinprojectmanagementwhilemaintainingCMMIcompliance.This is important because, one of the first steps to embrace change is to ensure that projectmanagementprocesses support and allow agility. In addition the people executing the process were trained as ScrumMastersbyJeffSutherland,whoalsodidamanagementseminaronScrumatSystematic.Concurrenttotheabove pilots, Leanwas considered by all projects and shared functions as one of severalways to identifypossibleimprovements.
Result.ThefirststepforSystematicinadaptingaLeanmindsetresultedintheadoptionofScrumandstorybaseddevelopmentastherecommendedwayofworking.Systematicprovidesadefault implementationofaProjectsDefinedProcess(PDP)knownasPDPCommon.ThePDPCommonhasbeenupdatedtointegrateScrumandstorybaseddevelopmentintotherelevantprocesses.
TheapparentresultofadoptingagilemethodsintoexistingCMMIcompliantprocesses,hasledtointegrationofprocessesorprocessareasthatinitiallywereimplementedseparately.Thenewprocessesaremoreefficient,andthechangeshaveimprovedquality,customerandemployeesatisfaction.
Risk.SomeoftherisksofapplyingAgilemindsettoaCMMIorganizationinclude:
• DegradingCMMIcompliantprocessestonon-compliance.
ãJeffSutherland1993-2010 132
• Localoptimizationsincreasingprojectefficiencyattheexpenseofinefficiencyattheorganizationallevel,e.g.duetolackoforganizationalcoordination.
Theseriskswerehandledbyacentralprocessteamthatkepttheorganizationontrackwithrespecttotherisksandchangemanagement.Theprocessteamwasresponsiblefor:
• BuildandsharecompetenciesonLean,AgileandScrumwiththeorganization.
• Defineandcommunicatevision,constraintsandmeasuresforadoptionofaLeanmindset.
• EncourageandempowerdifferentpartsoftheorganizationtochallengecurrentprocessimplementationswithaLeanmindset,insearchofimprovementopportunities.
• Collectexperiencesfromtheorganizationandconsolidateimprovementsattheorganizationallevel.
The dominant risk for failure in adapting Lean is insufficient understanding or adoption of Lean or Agilemindset.SystematichasaddressedthisriskbyinvitingJeffSutherlandandMaryPoppendiecktoSystematictoestablishagoodunderstandingofLean,ScrumandAgile.
SYSTEMATICEXPERIENCEFROMPILOTSIn a period of approximately 4months, two small projects piloted Scrumand early testing in story baseddevelopment.
Scrum.The first pilotwas initiated on a request for proposal, where Systematic inspired by Lean principlessuggestedadeliveryplanwithbi-weeklydeliveriesandstatedexplicitexpectationstocustomerinvolvementandfeedback.
OneofthemainreasonsthatSystematicwasawardedthecontractwasthecommitmenttodeliverworkingcodebi-weeklyandtherebyprovidingaverytransparentprocesstothecustomer.Duringprojectexecution,ahighcommunicationbandwidthwaskeptbetweentheteam,thecustomerandusers.Thiswasidentifiedasoneofthemainreasonsforachievinghighcustomersatisfaction.
The delivery plan and customer involvement resulted in early detection of technological issues. Had atraditionalapproachbeenusedtheseissueswouldhavebeenidentifiedmuchlaterwithnegativeimpactsoncostandscheduleperformance.
ãJeffSutherland1993-2010 133
However,productivityofthissmallprojectwasattheexpectedlevelcomparedtotheproductivityperformancebaseline for small projects. Another small project using Scrum shows a similar productivity and the sameindicationsonhighqualityandcustomersatisfaction.
AtSystematic,productivityforaprojectisdefinedasthetotalnumberoflinesofcodeproduceddividedbythetotalprojecteffortspent inhours.Thesenumbersaregathered fromtheconfigurationandversioncontrolsystem.Dataareattributedwithinformationrelatedtoprogramminglanguage,typeofcode:new,reuseortest.Thisdefinitionofproductivityhasbeenchosenbecauseitprovidessufficientinsightandisverysimpleandefficienttocollect.
Systematic has established and maintains a productivity performance baseline (PPB) for productivitycomparedtoprojectsizeestimatedinhours,fromdatacollectedoncompletedprojects[83].ThedatashowsthatproductivityishighonsmallprojectsanddeclineswiththesizeoftheprojectwithtraditionalCMMILevel5.Theproductivityperformancebaseline inSystematic isdividedintotwogroups:smallprojects lessthan4000hoursandlargeprojectsabove4000hours.Productivityofsmallprojectsis181%theproductivityoflargeprojects.
WhencomparingtheprojectsusingScrumtothecurrentproductivitybaselineitisseenthatproductivityforsmall projects is insignificantly changed, but the productivity for large projects shows 201% increase inproductivity. Asmentioned above, the large projects did additional improvements, and it is therefore notpossible to attribute the benefit solely to Scrum.However thepeople involved all agree that Scrumwas asignificantpartofthisimprovement.
There is a strong indication that large projects in Systematic using Scrumwill double productivity goingforward.SmallprojectsinSystematicalreadyshowahighproductivity.WebelievethatthisisbecausesmallprojectsinSystematicalwayshavebeenmanagedinawaysimilartoScrum.HoweverqualityandcustomersatisfactionseemstobeimprovedandwebelievethisisbecauseScrumhasfacilitatedabetterunderstandingofhowsmallprojectsaremanagedefficiently.
Earlytesting.Twodifferentapproacheswereusedtofacilitateearlytest.Onelargeprojectdecidedtouseastorybasedapproachtosoftwaredevelopmentandanotherprojectdecidedtofocusoncomprehensivetestingduringdevelopment.
Theideaofstory-baseddevelopmentwastosubdividefeaturesofwork,typicallyestimatedtohundredsofhoursofwork into smaller storiesof20-40hoursofwork.The implementationof a story followedanewprocedure,where the first activitywouldbe todecidehow the story couldbe testedbeforeany codewaswritten.Thistestcouldthenbeusedastheexitcriteriaforimplementationofthestory.
Inordertoensurethatthenewprocedurewasfollowed,theprocedureincludedafewcheckpointswhereaninspectorwouldinspecttheworkproduced,anddecidewhetherornotthedevelopercouldproceedtothenextactivityintheprocedure.Theseinspectionsarelightweight,andcouldtypicallybedoneinlessthan5minutes.
ãJeffSutherland1993-2010 134
Many benefits from story-based development were immediately apparent. The combination of a gooddefinitionofwhenastorywascomplete,andearlyincrementaltestingofthefeatures,providedaverypreciseoverviewofstatusandprogressforbothteamandotherstakeholders.
Developingaseriesofsmallstoriesratherthanpartsofabigfeatureismoresatisfactory,andcreatesabetterfocusoncompletingafeatureuntilitfulfillsallthecriteriaforbeing“done”.
Thisprojectfinishedearly,andreducedthenumberofcodingdefectsinfinaltestby38%comparedtopreviousprocesses.
Theprojectusingcomprehensivetestingplacedtestspecialiststogetherwiththedevelopers.Asinthestorybasedapproach,thiscauseddiscussionandreflectionbetweentesters,developers,userexperienceengineersandsoftwarearchitects,beforeorveryearlyinthedevelopmentofnewfunctionality.Asaconsequencetheamountofremainingcodingdefectsinfinaltestwerereducedby42%.
Basedonthesetwoprojectstestactivitiesshouldbeanintegratedactivitythroughouttheprojectslifetime.Scruminherentlysupportsthis,throughcross-functionalteamsandfrequentdeliveriestothecustomer.
Realneeds.Acustomersentarequestforproposalonafixedsetofrequirements.WhenSystematicresponded,weexpressedourconcernthatthescopeandcontentsexpressedintherequirementswerebeyondthecustomer’srealneeds.
Systematicdecided toopenly share the internalestimationof the requirementswith the customer, for thepurpose of narrowing scope by removing requirements not needed or too expensive compared to thecustomersbudget.Thecustomeragreedtore-evaluatetherequirementspecification,andtheresultwasthatrequirementsandpricewerereducedby50%.
This experience supports results in a StandishGroupStudy reported atXP2002by chairman Jim Johnson,showingthat64%offeaturesinafixedpricecontractareneverorrarelyusedbyend-users.
Webelievethatthisillustrateshowimportantitistohaveahighcommunicationbandwidthwiththecustomer,inordertofindoutwhattherealneedsare.Successisnotachievedbydoingthelargestproject,butbydoingtheprojectthatprovidesthemostvalueforthecustomer,leavingtimeforsoftwaredeveloperstoworkwithothercustomerswithrealneeds.Itgivesmotivationtodeveloperstoprovidesolutionstorealneed,whichinturnbenefitsdedicationandproductivity.
Eventhoughthisexperienceisrelatedtoactivitiesbeforetheprojectisstarted,thechallengeofmaintainingclosecommunicationwiththecustomer,toensurethattheprojectdeliversthemostvaluewithintheagreedconstraints,continuesandisstronglysupportedbyScrum.
ãJeffSutherland1993-2010 135
GUIDEFORMIXINGCMMIANDAGILETheprevioussectionhasdescribedhowSystematic,anorganizationappraisedtoCMMILevel5,hasadoptedagilemethods. This sectionpresents our advice to the agile organizations onhow to adopt the concept ofinstitutionalization.
HOWCMMICANIMPROVEAGILEOurfocusisonusingCMMItohelpanorganizationinstitutionalizeAgileMethods. WehaveallheardAgileMethodsdescribedbysomeasjustanotherdisguiseforundisciplinedhackingandofsomeindividualswhoclaimtobeAgilejustbecausethey“don’tdocument.”WebelievethevaluefromAgileMethodscanonlybeobtainedthroughdisciplineduse.CMMIhasaconceptofInstitutionalizationthatcanhelpestablishthisneededdiscipline.
Institutionalization isdefinedinCMMIas“theingrainedwayofdoingbusinessthatanorganizationfollowsroutinelyaspartofitscorporateculture.”Othershavedescribedinstitutionalizationassimply“thisisthewaywe do things around here.” Note that institutionalization is an organizational-level concept that supportsmultipleprojects.
CMMIsupportsinstitutionalizationthroughtheGenericPractices(GP)associatedwithallprocessareas.Forthepurposesofourdiscussion,wewilllookatthe12genericpracticesassociatedwithmaturitylevels2and3intheCMMI[72]pp.39-44andhowtheymighthelpanorganizationuseAgileMethods.Wehaveparaphrasedthegenericpractices(showninboldtextbelow)tomatchourrecommendedusagewithAgileMethods.InCMMIterms,theprojectsinanorganizationwouldbeexpectedtoperformanactivitythataccomplishedeachofthesegenericpractices.WehaveusedScrumastheexampleAgileMethodtodescribesomeoftheactivitiesthatrelatetothesepractices.
EstablishandmaintainanorganizationalpolicyforplanningandperformingAgileMethods(GP2.1).ThefirststeptowardinstitutionalizationofAgileMethodsistoestablishhowandwhentheywillbeusedintheorganization.AnorganizationmightdeterminethatAgileMethodswillbeusedonallprojectsorsomesubsetof projects based on size, type of product, technology, or other factors. This policy is a way to clearlycommunicatetheorganization’sintentregardingAgileMethods.InkeepingwiththeAgilePrincipleofface-to-faceconversionsat“allhandsmeeting”oravisitbyaseniormanagerduringaproject’skickoffcouldbeusedtocommunicatethepolicy.
EstablishandmaintaintheplanforperformingAgileMethods(GP2.2).ThispracticecanhelpensurethatAgileMethodsdonotdegradeintoundisciplinedhacking.TheexpectationisthatAgileMethodsareplannedand thatadefinedprocessexistsand is followed. Thedefinedprocess should includea sequenceof stepscapturingtheminimumessentialinformationneededtodescribewhataprojectreallydoes.Theplanwouldalsocapturetheessentialaspectsofhowtheother10genericpracticesaretobeimplementedintheproject.InScrum,someofthisplanningislikelytobecapturedinaproductbacklogand/orsprintbacklog,mostlikelywithinatoolasopposedtoadocument.
ãJeffSutherland1993-2010 136
Provide adequate resources for performing Agile Methods (GP2.3). Every project wants, needs, andexpects competent professionals, adequate funding, and appropriate facilities and tools. Implementing anactivitytoexplicitlymanagethesewantsandneedshasproveduseful.InScrum,forexample,theseneedsmaybereviewedandaddressedattheSprintPlanningMeetingandreconsideredwhensignificantchangesoccur.
AssignresponsibilityandauthorityforperformingAgileMethods(GP2.4).Foraprojecttobesuccessful,clearresponsibilityandauthorityneedtobedefined.Usuallythisincludesacombinationofroledescriptionsandassignments.Thedefinitionsoftheserolesidentifyalevelofresponsibilityandauthority.Forexample,aScrumProjectwouldassignan individualor individuals to the rolesofProductOwner,ScrumMaster,andTeam. Furthermore,therolesintheTeamarelikelytoincludeamixofdomainexperts,systemengineers,softwareengineers,architects,programmers,analysts,QAexperts,testers,UIdesigners,etc.ExpertiseintheTeam is likely to include a mix of domain experts, system engineers, software engineers, architects,programmers, analysts, QA experts, testers, UI designers, etc. Scrum assigns the team as a whole theresponsibility for delivering working software. The Product Owner is responsible for specifying andprioritizingthework.TheScrumMasterisresponsibleforassuringtheScrumprocessisfollowed.Managementisresponsibleforprovidingtherightexpertisetotheteam.
TrainthepeopleperformingAgileMethods(GP2.5).Therighttrainingcanincreasetheperformanceofcompetentprofessionalsandsupportsintroducingnewmethodsintoanorganization.PeopleneedtoreceiveconsistenttrainingintheAgileMethodbeingusedinordertoensureinstitutionalization.Thispracticeincludesdeterminingtheindividualstotrain,definingtheexacttrainingtoprovide,andperformingtheneededtraining.Trainingcanbeprovidedusingmanydifferentapproaches,includingprogrammedinstruction,formalizedon-the-jobtraining,mentoring,andformalandclassroomtraining.Itisimportantthatamechanismbedefinedtoensurethattraininghasoccurredandisbeneficial.
Placedesignatedworkproductsunderappropriatelevelofconfigurationmanagement(GP2.6).Thepurposeofaprojectistoproduceadeliverableproduct(orproducts).Thisproductisoftenacollectionofanumberofintermediateorsupportingworkproducts(code,manuals,softwaresystems,buildfiles,etc.).Eachof theseworkproductshasavalueandoftengoes throughaseriesofsteps that increase theirvalue. Theconceptofconfigurationmanagementisintendedtoprotectthesevaluableworkproductsbydefiningthelevelofcontrol,forexample,versioncontrolorbaselinecontrolandperhapsmultiplelevelsofbaselinecontroltousewithintheproject.
Identifyandinvolvetherelevantstakeholdersasplanned(GP2.7).InvolvingthecustomerasarelevantstakeholderisastrengthofAgileMethods.Thispracticefurtheridentifiestheneedtoensurethattheexpectedlevelofstakeholderinvolvementoccurs.Forexample,iftheprojectdependsoncustomerfeedbackwitheachincrement,build,orsprint,andinvolvementfallsshortofexpectationsitisthennecessarytocommunicatetotheappropriatelevel,individual,orgroupintheorganizationtoallowforcorrectiveaction.Thisisbecausecorrectiveactionmaybebeyondthescopeoftheprojectteam.InadvancedScrumimplementations,thisisoftenformalizedasaMetaScrum[41]wherestakeholdersserveasaboardofdirectorsfortheProductOwner.
ãJeffSutherland1993-2010 137
MonitorandcontrolAgileMethodsagainsttheplanandtakeappropriatecorrectiveaction(GP2.8).Thispracticeinvolvesmeasuringactualperformanceagainsttheproject’splanandtakingcorrectiveaction.Thedirectday-to-daymonitoringisastrongfeatureoftheDailyScrumMeeting.Further,examplesofthiscanbeseeninScrumwiththeuseoftheProductBurndownChartshowinghowmuchworkis lefttodoatthebeginningofeachSprintandtheSprintBurndownChartshowingthetotaltaskhoursremainingperday.ScrumenhancestheeffectivenessoftheplanbyallowingtheProductOwnertoinspectandadapttomaximizeROI,ratherthanmerelyassuringplanaccuracy.
ObjectivelyevaluateadherencetotheAgileMethodsandaddressnoncompliance(GP2.9).Thispracticeisbasedonhavingsomeonenotdirectlyresponsibleformanagingorperformingprojectactivitiesevaluatetheactualactivitiesoftheproject.Someorganizationsimplementthispracticeasbothanassuranceactivityandcoaching activity. The coaching concept matches many Agile Methods. The Scrum Master has primaryresponsibility for adherence to Scrum practices, tracking progress, removing impediments, resolvingpersonnelproblems,andisusuallynotengagedinimplementationofprojecttasks.TheProductOwnerhasprimaryresponsibilityforassuringsoftwaremeetsrequirementsandishighquality.
Review the activities, status, and results of the Agile Methods with higher-level management andresolve issues (GP2.10). The purpose of this practice is to ensure that higher-level management hasappropriatevisibilityintotheprojectactivities.Differentmanagershavedifferentneedsforinformation.AgileMethods have a high level of interaction, for example, Scrum has a Sprint PlanningMeeting, Daily ScrumMeetings,aSprintReviewMeeting,andaSprintRetrospectiveMeeting.ManagementneedsaresupportedbytransparencyofstatusdataproducedbytheScrumBurndownChart.This,incombinationwithdefectdatacanbeusedtoproduceacustomizedmanagementdashboardforprojectstatus.Managementresponsibilitiesareto(1)providestrategicvision,businessstrategy,andresources,(2)removeimpedimentssurfacedbyScrumteamsthattheteamscannotremovethemselves,(3)ensuregrowthandcareerpathofstaff,and(4)challengethe Scrum teams to move beyond mediocrity. The list of impediments generated by the Scrum teams istransparent tomanagement and it is their responsibility to assure they are removed in order to improveorganizationalperformance.
EstablishandmaintainthedescriptionofAgileMethods(GP3.1).ThispracticeisarefinementofGP2.2above. The only real difference is that description of Agile Methods in this practice is expected to beorganization-wide and not unique to a project. The result is that variability in how Agile Methods areperformed would be reduced across the organization; and therefore more exchange between projects ofpeople,tools,informationandproductscanbesupported.
CollecttheresultsfromusingAgileMethodstosupportfutureuseandimprovementtheorganization’sapproachtoAgileMethods(GP3.2).Thispracticesupportsthegoaloflearningacrossprojectsbycollectingtheresultsfromindividualprojects.TheScrumSprintRetrospectiveMeetingcouldbeusedasthemechanismforthispractice.
ãJeffSutherland1993-2010 138
Allofthesegenericpracticeshavebeenusefulinorganizationsimplementingotherprocesses.WehaveseenthatanumberofthesegenericpracticeshaveatleastpartialsupportinScrumorotherAgileMethods.WebelievethatimplementingthesepracticescanhelpestablishneededdisciplinetoanyAgileMethod.
CRITIQUESOFCMMInresearchfundedbytheDanishgovernment,Roseet.al.surveyedtheliteratureoncritiquesofCMM[84].Theyobserved that the chief criticismofCMM isnot theprocess itself, but the effectsof focusonprocessorientation. While side effects of process focus may be viewed as simply poor CMM implementation,organizationswithheavyweightprocessesarehighlypronetopoorexecution.
Aswithanyothermodel,goodandbadimplementationsofCMMexist.WebelievethatbadimplementationsareoneofthemainreasonsfortheexistenceofmanynegativecriticismsofCMM.Suchimplementationsareoften characterized as in the table below,whereasmany goodCMM implementations addressmost of thecriticism.
OnewaytoenhancechancesforagoodCMMorCMMIimplementationistouseScrum.ApplyingScrumandagilemindsetwhileimplementingCMMIwillhelptorecognizethatCMMIaddressespeopleandtechnologyaspects,inawayfullysupportiveofanagilemindset.
Moreimportantly,theworkinthispaperhasshownthatthemixofCMMIandScrumblendsamagicpotionforsoftwaredevelopmentthatisevenbetterthanthesumofthetwoalone.
WeacknowledgethattheCMMcriticismlistedinthetablebelowexist,butfromourknowledgeofCMMIweconsiderittobeincorrect.ButabadimplementationofCMMImaybeperceivedthisway.EventhoughgoodCMMIimplementationscanbedonewithoutagilemethods,thetableshowsthatScrumwillcontributewithabeneficialfocusonissuesstemmingfrom“bad”CMMIimplementation.
ãJeffSutherland1993-2010 139
CMMcriticism Scrumsupport
CMMreveresprocessbutignorespeople. Scrum is the first development process to treatpeopleissuesthesameasotherprojectmanagementissues[85].
Does not focus on underlying organizationalproblemsthatshouldbesolved.
Aprimary responsibilityof theScrumMaster is tomaintain and resolve an impediment list thatcontainsorganizationalissues,personalissues,andtechnicalproblems.
Ignoresqualityinthesoftwareproductassuminganunproven link between quality in the process andqualityintheresultingproduct.Differingprojectandorganizational circumstances may mean that aprocessthatdeliversagoodproductinonecontextdeliversapoorproductinanothercontext.
The Scrum Product Owner is responsible forcontinuously reprioritizing the Product Backlog tomaximizebusinessvalueincurrentcontext.
Lackofbusinessorientation TheprimaryfocusofScrumisondeliveringbusinessvalue.
Poorawarenessoforganizationalcontext. Creation and prioritization of features, tasks, andimpediments is always done in organizationalcontextbyinspectedandadapting.
Ignorestechnicalandorganizationalinfrastructures. Daily inspectionandadaptation inScrummeetingsfocusesontechnicalandorganizationalissues.
Encourages an internal efficiency focus and thusmarketandcompetitionblindness.
Focusisondeliveringbusinessvalue.TypeCScrumallows an entire company to dominate a marketsegment through inspecting and adapting in realtimetocompetition[41].
CONCLUSIONSThispapershowsthatCMMIandScrumcanbesuccessfullymixed.ThemixresultsinsignificantlyimprovedperformancewhilemaintainingcompliancetoCMMILevel5ascomparedtoperformancewitheitherCMMIorScrumalone.
Scrum pilot projects showed significant gains in productivity and quality over traditionalmethods. Theseresults led toanROIbaseddecision tomorewidely introduceScrumandconsiderotherAgilepractices inSystematic. Scrumnowreducesevery categoryofwork (defects, rework, totalwork required, andprocessoverhead)byalmost50%.
ThispaperhasoutlinedhowSystematicadoptedScrumandstorybaseddevelopmentintoitsCMMIprocessesinspiredfromastrategicfocusonLean.ForAgilecompaniesthearticlehaspresentedhowGenericPractices
ãJeffSutherland1993-2010 140
fromCMMIcanbeusedtoinstitutionalizeagilepractices.FurthermorethearticlehaspresentedLeanSoftwareDevelopment[8]asanoperationaltooltoidentifyimprovementopportunitiesinaCMMI5company.
Wethinkcompaniesindefense,aerospace,andotherindustriesthatrequirehighmaturityofprocesses,shouldcarefullyconsiderintroducingAgilepracticesintotheworkplaceandallsoftwarecompaniesshouldconsiderintroducingCMMIpracticesintotheirenvironment.
OurrecommendationtotheAgilecommunityistousetheCMMIgenericpracticesfromCMMILevel3toamplifythe benefits from Agile methods. The efficiencies of agile practice can lower the cost of CMMI processimprovementsmakingthebenefitsmorewidelyavailable.OurrecommendationtotheCMMIcommunity isthat Agile methods can fit into your CMMI framework and will provide exciting improvements to yourorganization.
ãJeffSutherland1993-2010 141
MATUREAGILEWITHATWISTOFCMMI
CarstenRusengJakobsen
SystematicSoftwareEngineering
KentAaronJohnson
AgileDigm,Incorporated
ABSTRACTSystematicisanagilecompanyworkingatCMMIlevel5,wherethedefaultwayofworkingisbasedonScrum
and storybasedearly testingdevelopment. Solidexperiences in combiningCMMIwithScrumand storybaseddevelopment,hasshownthatthemixprovidesstrongsynergies[2]andinsightsintowhatCMMIpracticesfitandamplifytheexecutionofScrumandstorybasedearlytestingdevelopment
Thispaperpresents specificallyhowagilemethods likeScrumare successfully combinedwithCMMI.CMMIprovidessolidsupportforwhatdisciplinestoconsider.Whenappliedthedisciplinescreateafocusonimportantaspectsofagilemethodsthatperhapsarenotnormallyelaborated,forexamplehowtoensureaproperqualityofaproductbacklogorhowtoensureaproper“productionline”fortheproject.Thisguidancemaynotbeneededforsmallagileprojects,butastheagilemovementcontinuestogrow,andisusedforlargerandmorecomplexprojects,agileprojectswillneedtoaddresstheseissuesrelatedtoincreasedsizeandcomplexity.
TheexperiencesfromcombiningCMMIandScrumhaveledSystematictoidentifyexamplesofexplicitguidancefromCMMIthathelptoexecutenormalScrumactivitiesevenbetter.
Theseactivitiescanbeimplementedinthespiritoftheagilemanifestoandprinciplesandbydoingsoagilemethodscanbeaugmentedandmaturedtoensurethatevenlargerandmorecomplexprojectsinthefuturecanandwillbenefitfromagile-withatwistofCMMI.
INTRODUCTIONThispaperpresentstheexperiencesonhowCMMIamplifiesAgileandrecommendsasubsetofactivitiesanagileprojectcouldadoptfromCMMItoimproveperformance.Agilepuristsandsmallagileprojectsmayfindtheseactivitiesnon-agileorcounter-productive;howeverinlargerand/ordistributedprojectstheseactivitieswillprovetobeinvaluable.
The paper does not describe how tomature an organization from CMMI level 1 to CMMI level 2, but theactivitiesdescribedcouldbepartofsuchachange.RatherthispaperhighlightsasetofactivitiesthatcouldbeconsideredthegluebetweenagileonlyprojectsbasedonScrumandthedisciplinesexpectedfromprojectsandorganizationsworkingtowardCMMIlevel2or3.
Thispaperpresentshow“atwistofCMMI”canhelpestablishamoresolid frameworkforagileprojects tosupportevenmorecomplexprojectsbyadoptingsomeofthepracticesfromtheCMMI.
ãJeffSutherland1993-2010 142
CONTEXTFORTHEEXPERIENCESSystematicwasestablishedin1985andemploysmorethan450peopleworldwidewithofficesinDenmark,Finland,USAandtheUK.ItisanindependentsoftwareandsystemscompanyfocusingoncomplexandcriticalITsolutionswithininformationandcommunicationsystems.Oftenthesesystemsaremissioncriticalwithhighdemandsonreliability,safety,accuracyandusability.
Customers are typically professional IT-departments in public institutions and large companies withlongstandingexperienceinacquiringcomplexsoftwareandsystems.SolutionsdevelopedbySystematicareused by tens of thousands of people in the defense, healthcare, manufacturing, and service industries.Systematicwas appraised 11 November 2005 using the SCAMPISMmethod and found to be CMMI level 5compliant. During 2006 Systematic adopted Scrum and a story based early testing approach to softwaredevelopment.
CMMIprovidesinsightintowhatprocessesareneededtomaintainadisciplinedmatureorganizationcapableof predicting and improving performance of the organization and projects. Scrum provides guidance forefficientmanagementofprojectsinawaythatallowsforhighflexibilityandadaptability.Whenmixingthetwo, a magic potion emerges, where the mindset from Scrum ensures that processes are implementedefficientlywhileembracingchange,andCMMIensuresthatallrelevantprocessesareconsideredwithproperdiscipline.
IndividuallyCMMIandScrumhasprovenbenefits,butalsopitfalls.AnagilecompanymayimplementScrumcorrectly,butfailtoobtainrealbenefitsduetolackofconsistentandsufficientexecutionofengineeringormanagementprocesses.CMMIcanhelpagilecompaniestoinstitutionalizeagilemethodsmoreconsistentlyandunderstandwhatprocessestoaddress.
AcompanycancomplywithCMMI,butfailtoreachoptimalperformanceduetoinadequateimplementationofprocesses.ScrumandotheragilemethodscanguidesuchcompaniestowardsmoreefficientimplementationofCMMIprocessrequirements.
SystematichasgainedvaluableexperiencesincombiningScrumandCMMIthatarerelevantbothforprojectsinaCMMIandagilecontext.
EXPERIENCESFROMMIXING
SMCapabilityMaturityModelIntegration,andSCAMPIareservicemarksofCarnegieMellonUniversity
ãJeffSutherland1993-2010 143
ThefirstmajorexperiencefromworkingwithScruminaCMMIcontextisthatCMMIembracesScrum.CMMIhasmorepracticesandsupportforinitialprojectplanningandforfinaldeliveryandprojectclosure.InScrumterms,CMMIsuggestsactivitiesbeforeandaftersprintsareexecutedontheproductbacklog.
FromaCMMIperspective, the initialScrumproductbacklog iscreatedduringprojectplanningandduringprojectexecution;sprintsareexecutedandtheproductbacklogisupdated.Thislogicallysplitsplanningintotwoparts:overallCMMIprojectplanninganddetailedagileplanningthroughScrum.
Thisseparationhasledtooverallplanningwhereworkisplannedinsufficientdetailasopposedtoacompletedecomposition.TheoverallplanningproducesasetofoverallprojectplansandaScrumproductbacklogwhereacompletelistofprioritizedfeaturesorworkfortheprojectismanaged.
Theprimarychangetoprojectexecutionprocesses,wastointegrateScrumasthemethodforcompletingsmalliterations(sprints),onaselectedsubsetoftheworkwithhighestpriority.
SystematicexperienceindicatesthatthismixofCMMI,Scrum,andagileisbeneficial,because
• CMMIplanningcanbeconsideredakindofdisciplinedsprintzero,whereitisensuredthatanoptimalframeworkfortheprojectisestablished,includingahighqualityproductbacklog,aproductionlinedefinition,andwellknowntargetsandvisionfortheprojectasawhole.
• CMMIriskmanagementproactivelyaddressespossibleimpedimentsbeforetheyareencounteredbytheteam.
• CMMIqualityplanningspecifiesmoreaccuratelyandefficientlythequalitytargetsoftheprojectandhelpsdeveloperstoabetterinterpretationofcompletioncriteriaandsprintgoals.
• CMMIwillensurethattheprojectistrackedasawholeallowingtheScrumTeamtoconcentrateoncurrentsprint,knowingthattheyperiodicallyareinformedofoverallprojectstatus.
• Scrumrequiresdisciplineregardingautomatictest,anightlybuild,andintegration.CMMIsupportsthisneedfordisciplineandhasledsomeprojectsatSystematictomonitor“fix-timeafterfailedbuilds”formore than a year. Themeasure has proven to be cheap to establish, easy to understand, andthereforefacilitatinggoodhabits.
• CMMIexpects theproject to seekobjectivemeasuresofperformanceof theproject’sprocesses. InScrumprogress(ofsprints)isprimarilymeasuredthroughthesprintburndownchartandthesprintreviewmeeting.Theprojectmanagertrackstheprojectasawholebasedonselectedmeasureswithinkeyareaslike,productsize,earnedvalue,schedule,andquality.CMMIprojectplanningprovidesgoodoverallplansforthecompleteprojectwhereeachcompletedsprintisveryvaluableinput.
• CMMIensuresthatagilemethodsareinstitutionalized,including• Consistentimplementationthroughouttheorganizationandcontinuousimprovement,e.g.Systematic
ScrumGuidelines,storyinspectionchecklist.• Rolebasedtrainingofallroles,e.g.ScrumMasterandProductOwner.
ãJeffSutherland1993-2010 144
WhenSystematicadoptedScrum,therolesScrumMaster,ProductOwner,andTeamwereintroduced.Mostprojects inSystematichaveapersonwhocommunicatesandunderstands thecustomer. InSystematic thispersonisappointedtheroleProductOwner. InourexperiencetheProductOwner isoftenalsotheProjectManager,butcouldalsobeSoftwareArchitect,orUserExperienceEngineer.TheScrumMastersareinmostcases equivalent with the team leader roles in Systematic. Projects in Systematic are staffed with peopleworkingfulltimeontheprojectandtheteamisco-located.
HOWTOESTABLISHABETTERINITIALPRODUCTBACKLOGExperience.ProjectPlanninginCMMIisadisciplinedandcomprehensiveSprintZero.
InordertorunagoodScrum,itisvitaltohavegoodproductbacklog.WhenSystematicadoptedScrum,theprojectplanningprocesswasupdatedtoproduceaninitialproductbacklog.ExpectedCMMIpracticesincludedecompositionofworkintomanageablepiecesthatareestimatedandanalyzedfordependencies,planningofstakeholderinvolvement,andtotalprojectriskassessment.
Inadditiontotheproductbacklogasetofoverallprojectplansareestablished.Agileteamstalkaboutasprintzero to establish the foundation for the team to do efficient sprinting. Project Planning in CMMI can beperceivedasasprintzerotoproduceacoherentsetofplans,thatwillhelpimproveexecutionoftheproductbacklog. Such plans cover topics like, stakeholder management, milestone and delivery schedules, costestimates,andquality.
Theinitialversionoftheseplansaretypicallyestablishedwithinfewweeksafterprojectinitiationandwillfocusonthemostcertainelementsoftheprojectsplan,leavingmoreuncertainpartstobemanagedontheproductbacklog.
Figure1showsthemainactivitiesperformedtoestablishtheprojectplan.Thesequenceshowninthefigureisadvisory. The same activity may be performed multiple times, and often many activities are performedsimultaneously. Each of the activities are supported by short step-wise descriptions of how the activitynormallyisperformed,whatinputsarerequiredandwhatoutputareproduced.
ThedescriptionsarebasedonlessonslearnedandbestpracticesfromallprojectswithinSystematic,andareorganizedwithshortoutlineddescriptionsatthetop,anddetailedguidelinesandtemplatesatthebottom.
Thisprovides the teamwithsolidsupport forestablishing theprojectsplans.Theamountof timespent toestablishprojectplansvaries fromproject toproject, but formostprojects, initialprojectplanning canbecompletedwithinweeks.OneofthereasonsforthisisthehandlingofuncertainscopethroughScrumcombinedwithproperriskmanagementfromCMMI.
ãJeffSutherland1993-2010 145
Concurrent to the initial planning, the overall solution architecture and product quality objectives areelaboratedanddocumentedbysoftwarearchitectsandleaddevelopers.
Before theprojectmoves fromplanning (sprint zero) toproject execution (sprinting) theprojectmanagervalidates that the overall project objectives and plans are achievable and realistic, and that planning hasreducedprojectriskssufficiently.
Manyoftheactivitiesinfigure1areinlinewiththeintensionsofScrum,themaindifferenceisthatinCMMItheseactivitiesareelaboratedanddocumented.
ãJeffSutherland1993-2010 146
FIGURE13OVERVIEWOFCMMIPLANNINGACTIVITIES
Experience.Twolevelsofplanningandtracking:Projectasawholeandeachsprint.
Sprintingontheproductbacklogisstarted,whentheprojectplansandsolutionarchitectureareapprovedbyseniormanagement.
Fromthispointtrackingoftheprojectisdoneattwodifferentlevelsconcurrently.Theprojectmanagertrackstheprojectsoverallplansandteam(s)tracksprogressofactivesprint(s).
[Project Plan Approved]
[Project Approved]
Project Manager
Plan RiskResponse
DefineScope
[Project Scope and Objectives Approved]
Create WBS
EstimateCost
Create Product Backlog
PlanOrganization
PlanQuality
Plan RiskManagement
DevelopSchedule
BudgetCost
IdentifyStakeholders
PlanCommunications
Analyse RisksQuantitatively
Analyse RisksQualitatively
IdentifyRisks
ReleaseProject Plan
Develop ProjectManagement Strategy
DevelopProcess
PlanMeasurement
Establish ChangeManagement
Establish Configu-ration Management
Plan Purchase and Acqusition
PlanContracting
IdentifyConfiguration Items
PlanProcess
Plan Milestonesand Deliveries
ãJeffSutherland1993-2010 147
Theplansestablishedwithplanningactivitiesprovidesabettercontextfordefiningsprintvisionsandgoals,comparedtoprojectsonlyusingaSprintbacklog.Theyalsoallowtheteamtofocusonthesprint,becausetheycanrelyontheprojectmanagertrackingtheoverallprogress.
RISKSANDIMPEDIMENTSExperience:Planningandriskmanagementactivitiesreducesriskofproductbacklog
The firstdraftproductbacklog isassessed forriskby the team. Asking the teamtoestimate theproductsbacklog using 3-point estimates for effortwill reveal themost uncertain parts of thework in the ProductBacklog. Conducting Risk Identification meetings will identify other important risks, and allow proactivemitigationtobeinitiated.
Experience:Riskmanagementcanproactivelypreventimpediments
Scrum has a strong focus on removing impediments as soon as they are identified, however CMMI riskmanagement activities focus on proactively identify some of these impediments as risks, and throughmitigationeliminatethembeforetheyoccurasanimpedimentinthefuture.
The distinction between risk and impediment is that risk describe a problem thatmay occur,whereas animpedimentisproblemthathasoccurredandisimpactingplannedprogress.
Riskmanagementactivitiesareeasily integratedwithScrumactivities.Duringprojectplanning theprojectplansandsolutionarchitectureareinputsforinitialidentificationofrisks.Duringprojectexecution,bi-weeklymeetings of 10-15minutes are arranged, where the status of known risks is reported and new risks areidentified.
Itisourexperiencethattheseriskmanagementmeetingsshouldbekeptoutsidethedailyscrummeeting.Newrisksmaybereportedonthedailyscrum,inwhichcasetheriskmanagerwilljusttakeanote.
HOWTOENSUREHIGHQUALITYScrumisdesignedtoproducehighqualityintermsofperceivedandconceptualintegritywhereshortiterationsandsprintreviewcustomersaremaindriversforhighquality.
Experience:Explicitqualityplansimproveshelpstheteamtobuildtherightqualityin
ãJeffSutherland1993-2010 148
Oneoftheresultsofplanningisaqualityassuranceschedule(QAS),whereitisoutlinedwhatqualityactivitieswillbeusedtoensurethequalityobjectivesareachieved.TheQASmayspecify
• Whatstoriesaresubjecttoinspection
• Whatcodeissubjecttoreview
• Whatdocumentsaresubjecttowhattypesofreview
• Whatunittestandautomatictestisproduced
• Whatisincludedintheacceptancetest
AtypicalQASdocumentisonlyafewpageslong,buttheabovedescriptionscanhelpascrumteamtoelaborateandunderstandthedefinitionofdone.
Experiece:Usechecklisttoensurequalityofstories
OneoftheimportantaspectsofSystematicstorybaseddevelopmentmethodwastoensurefocusonearlytest.
ãJeffSutherland1993-2010 149
FIGURE14STORYINSPECTIONCHECKLIST
Thequalityofstoriesaregenerallyensuredbyfocusonearlyspecificationoftestandbygettingsomebodyelsetolookattheworkdone.
Developingastoryincludesmanydifferentactivities,thatneedtobestructuredtosomedegree.
TheStoryCompletionChecklistaccomplishesthesegoals,bystructuringactivitiesanddefiningwhenworkmustbeinspectedbyaninspector.
Theactivitiesinthechecklistallhaveashort5-10linedescriptioninaprocedurecalled“Executestory”,thatclarifieswhyandhowtheactivityisperformed.
ãJeffSutherland1993-2010 150
The inspector role is often appointed to a lead developer, and this way the inspection also serves as anopportunityforknowledgesharingbetweenexperiencedandlessexperienceddevelopers.
TheStoryCompletionChecklistensuresqualityatthestorylevelandmakesiteasierforthedeveloperatthesametime.
TEST,INTEGRATION,RELEASEANDCONFIGURATIONMANAGEMENTScrum promotes short iterations, e.g. one sprint permonth, and this in turn drives the need for efficientconfigurationmanagement,test,integrationandrelease.
CMMIhelpswith:
• Establishstandardsforproductionline,includingstandardsetupsforbuild-andtestservers
• Establishdisciplineoncriteriaforintegration
• Measurestoobjectivelyevaluateperformance
• Disciplinestomaintainintegrityofconfigurationmanagementsystem,builds,andreleases.
Experience:Automatedtestisamustinordertodoonemonthsprints
Whenthesprintdurationisonemonth,alltestsmustbeautomatedtotheextentpossible.Itisanintegratedpartofdevelopingastory,toalsoimplementtheautomatedtestverifyingthestory.Automatedtestareusedontheteamssharedrepositoryandruneverytimeadevelopercommitscodetothesharedbuildserver.
Experience:Automatedtestandintegrationmustbesupportedbyastandardproductionline
Everyprojectneedsthis infrastructureandthereforewehaveestablishedstandardproduction linesetups,allowingprojectstogetstartedfaster.
Experience:Continuousintegrationmustbesupportedwithdisciplineforcheckin
Inordertoavoidchaoswhendeveloperscontinuouslyintegratewitheachother,wehavedefinedthefollowingcriteriaforcheckinofcodetotheintegrationrepository:ThetestmustrunsmoothlyinthedeveloperssandboxandthecodemustcomplywiththecodestandardcheckedwithFxCop(astaticcodeanalysistool).
ãJeffSutherland1993-2010 151
Experience:Focuson“fix-timeafterfailedbuild”drivesgooddisciplineinproject
Usingstandardinfrastructuresetupsallowsforefficientdatacollectionandanalysis.InparticularSystematichas been inspired from Lean thoughts on flow and jidoka (stopping the production linewhen an error isdetected).
Wewantourprojectstobeabletodeliveronadailybasis,andhencethatunresolvedfailedbuildsarefixedwithinaworkingday.WeuseCruiseControl(abuildmanagementtool)tosignalalldeveloperswhenabuildfails.Wealsomonitortheobjectivebyanalyzingdatafromthebuildserversusingcontrolchartsliketheoneshownbelow.
Manyprojectshaveachievedthisoneworkdayobjective,merelybythefocusonthemeasure.Theobjectiveiseasytounderstand,andpresentingtheinformationinCruiseControlandcontrolchartshasestablishedagoodhabitoffixingbrokenbuildsimmediatelywhentheyfail.
Experience:PeriodicauditofConfigurationManagementsystembuildsgoodhabits
Aspartofeverysprintdeliveryaworkproductevaluation(WPE)isconductedandforeverydeliverytothecustomerafunctionalconfigurationaudit(FCA)isconducted.Thepurposeistoensurethatthebuildproductiscorrectandcomplete.
WPEandFCAareexecutedanddocumentedbyfillingoutcheck-liststhathelpstoensurethatconfigurationmanagement activitieshasbeenexecuted correctly for thebuildor release.Usually these activities canbeaccomplishedwithinoneortwohours.
Fix time after failed build
0,00
2,00
4,00
6,00
8,00
10,00
12,00
233
238
243
248
253
258
263
268
273
278
283
288
293
298
303
308
313
318
323
328
Build ID
Hour
s
Fix Time (Hours) LCL of avg fix time Avg fix time UCL of avg fix time
ãJeffSutherland1993-2010 152
Theexperienceisthatthischeckingoftheconfigurationmanagementsystemonamonthlybasis,buildsgoodhabitsontheteamtorememberconfigurationmanagementdisciplines,andasaresultbuildsandreleasesarecompleteandcorrect,andmaybere-createdinthefutureshouldtheneedarise.
AGILEWITHATWISTOFCMMIIn[2],[3]wedescribedhowthegenericpracticesfromCMMIcanbeusedtoinstitutionalizeScruminyourorganization.Inthefollowingwepresentourrecommendationstoactivitiesanagileprojectcouldconsideradopting.TheseactivitiesareinspiredbythemandatorygoalsandexpectedpracticesfromasubsetofCMMIprocessareas.
Werecommendthefollowingactivitiestoagileprojects:
1. Establishyourownsprintzero,andincludeactivitiesinitem2-6belowinit.
2. UseRiskManagementtoproactivelyaddressrisksbeforetheyareidentifiedasimpediments
3. Decompose requirements into features on the product backlog. Prepare the product backlog bydecomposing the highest prioritized features to stories allowing for efficient sprint planning. (Thisdefineswhatyouarereallygoingtodo.)
4. Use3-pointeffortestimatesonelementsoftheproductbacklogduringinitialplanning.
5. Analyzedependencies,stakeholders,riskonelementsofproductbacklog.
6. Establishmilestoneanddeliveryplanandtheirinitialrelationshiptoproductbacklog.
7. UseStoryCompletionChecklisttomaintainhighqualityofstoriesproduced.
8. Decideandcommunicatequalityobjectivesincluding,whatcodeanddocumentationtoformallyreviewtoelaboratedefinitionofdone.
9. Establishstandardsforproject“productionline”includingdevelopment,buildservers,andtestservers.
10. Automatetestandnightlybuild,andmeasureperformance.
11. Establishcriteriaforcommittingofcodetointegration.
12. Maintainintegrityofconfigurationmanagement,byusingachecklistforWorkProductEvaluationandexecuteitbytheendofeachsprint.
CONCLUSIONThispaperpresentedasomepracticaladviceforagileprojectsonadditionalactivitiestoadoptparticularlyinlargerordistributedprojects.
ãJeffSutherland1993-2010 153
OurrecommendationtotheAgilecommunityistoextendagilemethodsinspiredfromanunderstandingofthemandatorygoalsandexpectedpracticesforCMMIlevel2and3.Thesepracticesmakegoodsense,andyoucouldarguethat ithasalways inherentlybeenexpectedaspartofyouragilemethod. Ingeneral theCMMImodelprovidesagoodunderstandingwhatpracticestoconsider–butyouwillhavetoadoptittoyourcontext,andfindagileimplementationsforthepractices.
Whenprojectsgrow,webelieveyouneedmorediscipline.Wehavedescribedhowamoredisciplinedsprintzero,riskmanagement,andvariouschecklistswithminimaleffortcanbringyouslightlymoredisciplineintoyourproject–andwebelievethatdoingsowillbringsuccesstolargerordistributedagileprojects.
REFERENCES[1]M. Poppendieck andT. Poppendieck, Lean SoftwareDevelopment:An ImplementationGuide:Addison-Wesley,2006.
[2] J. Sutherland,C.R. JakobsenandK.A. Johnson, "CMMIandScrum -amagicpotion for codewarriors" inproceedingsforAgile2007
[3]M.K.KulpaandK.A.Johnson,InterpretingtheCMMI:AProcessImprovementApproach,SecondEdition.BocaRaton:AuerbachPublications,2008
ãJeffSutherland1993-2010 154
CHAPTER5:SCRUMMETRICS
Fromthebeginning,Scrumwasdesignedtoprovidetransparencytobothteammembersandthoseoutsidetheteam.AtypicalScrummaintainsaScrumboardshowingcolumnsofuserstories,developmenttasksrelatingtoeachstory,andthestateofeachdevelopmenttaskandtestsassociatedwitheachstory.Whenataskisstartedacardismovedacrosstheboardintoanopencolumn.Whencodeiscompleteitismovetotheverificationcolumnandwhenfullytestedismovedtoadonecolumn.AnupdatedScrumburndownchartalongwithaprioritized listof impediments isoftenpostedaswell. SomesitesevenhaveLava lamps that changecolordependingonthestateofthebuildorstateoftheSprint.
AmanagercanwalkbyaScrumboardandseethestateoftheteaminafewseconds.Ifanimpedimentslistisposted,amanagercanaddrelevant items tohisorherwork list.Thiseliminates theneed formost statusreporting,particularlyifcriticalinformationisputonlineonawebpage,awiki,orareportingtool.
ManagingandtrackingcoststoevaluateactualReturnonInvestment(ROI)canprovideanadditionalfeedbackloop.EarnedValueManagement(EVM)maybeofinteresttomanagement.Sulaiman,Barton,andBlackburn[86]providedetailedcalculationsforEVMandevaluationofitseffectivenessinanAgile2006researchpaper.
ãJeffSutherland1993-2010 155
REPORTINGSCRUMPROJECTProgressTOEXECUTIVEMANAGEMENTTHROUGHMETRICS
BrentBarton,KenSchwaber,DanRawsthorne2005
Contributors:FrancoisBeauregard,BillMcMichael,JeanMcAuliffe,VictorSzalvay
INTRODUCTION
TheinterestinAgilesoftwaremethodologiesisnotsurprising.Agilemethodsarepresentinganopportunitytodevelopsoftwarebetterandthisisbeingnoticedinthebusinesscommunity.ScrumisparticularlyofinterestpartlybecauseofitsROIfocusandquickimplementation.WhiletheeffortsofinnovatorsandearlyadoptershavehelpedusassertthatAgileisbetterthantraditionalmethods,improvingthereportingcapabilitywouldhelp.Evenbetterwouldbeabletoreportprojectprogresstoexecutivemanagementinamorecompellingway.AtaScrumGathering,whitepapersweresubmittedanddiscussed.Thisisasummaryofthosediscussionsandtheintegrationofthecontributionsofmanypeople.Visibilityintoprojectprogressandproject“health”isaconsistentthemeexecutivemanagementdesires.
TRANSPARENCYINTOPROJECTS
ExecutiveManagementneedstransparencyintoalloperationsbyviewingimportantindicatorsquickly:Thisisespeciallytrueofsoftwareprojects.Theywantnosurprisesbecauseinsoftwareasurpriseisrarelyapleasantone.Itisworthmentioning,however,thatbadthingsdohappen;executivesknowthisandsodoeseveryoneelse.Itisalwaysasurprisethefirsttimeonehearsbadnews.Incontrast,thekindofsurpriseexecutiveshatethemost,havesignificantimpactandwereknownmuchearlierthanwhentheywerefinallyinformed.Thenegativeemotionalresponsetothesurpriseisreinforcedbytherealizationthatdecisionsweremadeonfaultyinformationandthiswaspreventable.
Therearemanytechniquesandpracticesforassessingtheprogressandprobablesuccessofprojects.Scrumprovides four simple and effective artifacts for managing and monitoring project performance: ProductBacklog,ProductBurndown,SprintBacklogandSprintBurndown. Buildingon these,weare integratingaFunctionalWorkBreakdownStructureandatechniqueformeasuringEarnedBusinessValue.
Stakeholdersandexecutivesoftenhaveparticularinterestincertainareasofprojects.ThegroupingnatureofaWorkBreakdownStructure(WBS)affordstheopportunitytopresentprogressatamid-level:notasingleviewlikeaburndownandnotatadetaillevellikeabacklog.BycombiningaWBS,transparencycanbeattainedquicklywithafewsimple,graphicalreportsonanexecutivedashboard
ãJeffSutherland1993-2010 156
EXECUTIVEDASHBOARD
The Executive Dashboard presented here is easily read, interpreted and provides the ability to referenceadditionalmaterialifdesired(seeFigure2:ExecutiveDashboard).
ãJeffSutherland1993-2010 157
FIGURE15:EXECUTIVEDASHBOARD
Thecontentsofthisdashboardreportinclude:
ParkingLot: This isapictorial thatstatusesgroupsof featuresorusecases. ThishasbeenadoptedfromreportsfoundinFeatureDrivenDevelopment(FDD).WiththeadditionofaBusinessValueIndex(describedlater),onecansee theprogressandvalueof thisarea to thebusiness. Ataglance, thecolorsshowwhereprogressismade,areasofconcernareanditemsnotstarted.TheBVIrepresentsthetotalvalueoftheproject
ATM Project Dashboard
Parking Lot
LEGEND Work Breakdown Structure
CompletedAttentionIn ProcessNot Started
Progress Bar
Business Value
BVI Owner
Feature Group,Use Case Package
or Use Case
26%
25%
Business Support Activities
KS
( 17% )
( 6 )
17% 12%
Login Withdraw Cash Deposit Check
KS BB DR
# features or stories
% complete
Planned Completion
( 0% )
March 2005
17% 2%DR
Transfer Funds Buy Stamps
( 3 )
February 2005
( 5 )
( 100% )
January 2005
( 4 )
( 40% )
Planned Completion
BB
( 32% )
March 2005
( 2 )
( 0% )
March 2005
( 7 )
Product Burndown
-1000
-500
0
500
1000
1500
2000
2500
3000
0 1 2 3 4 5
Sprint
Wor
k
Estimated CompletionMay 3, 2005
ATM
3 - Product 0 - Team 1 - Business
1- Function 0 - Structure
15 - Login
10 - Withdraw Cash7 - Deposit Check10 - Transfer
1 - Buy
Conversions
Rewrites
Refactoring
Management
Team Training
Dev/SCM/Test Dev Process
App
Tools
Maintenance
2 - Sales
1 - Marketing
1 - User
2 - User Docs
1 - Business
1 - Adapt
Space for: Links Risks Issues Highlights
$-
$100,000
$200,000
$300,000
$400,000
$500,000
$600,000
0 1 2 3 4 5
Sprint
Bus
ines
s Va
lue
ãJeffSutherland1993-2010 158
and theowner’s initialsdescribewho is responsible for thegroups. The legend is included(seeFigure3:ParkingLot).
Figure16:ParkingLot
Product Burndown: The burndown in work budgeted and planned compared as decreased by workcompletedacrosstime. Baseduponthis,anestimatedcompletiondatecanbedeterminedasthetrendlinecrossesthex-axis(seeFigure4:ProductBurndown).
bbb
Figure17:ProductBurndown
EarnedBusinessValueGraph:ThispresentstheBusinessValueearnedcomparedtothePlannedBusinessValue.VariancecanbequicklyestimatedfromthegraphtoassessthecorrectprioritizationandprogressoftheprojectFigure5:BusinessValueBurnup).
LEGEND
CompletedAttentionIn ProcessNot Started
Progress Bar
BVI Owner
Feature Group,Use Case Package
or Use Case
26%
25%
Business Support Activities
KS
( 17% )
( 6 )
17% 12%
Login Withdraw Cash Deposit Check
KS BB DR
# features or stories
% complete
Planned Completion
( 0% )
March 2005
17% 2%DR
Transfer Funds Buy Stamps
( 3 )
February 2005
( 5 )
( 100% )
January 2005
( 4 )
( 40% )
Planned Completion
BB
( 32% )
March 2005
( 2 )
( 0% )
March 2005
( 7 )
ãJeffSutherland1993-2010 159
Figure18:BusinessValueBurnup
GraphicalWorkBreakdownStructure:Thisvisualrepresentationprovidesaconcise,high-levelpresentationoftheprojectworkitems(seeFigure6:FunctionalWorkBreakdownStructure).
Spaceforlinks,highlights,issuesandrisks.Everyprojectandcustomerhasitsownspecificneeds.Thisspaceisintendedforafewbulletpoints.
WORK BREAKDOWN STRUCTURE
Dan Rawsthorne introduced a functional Work Breakdown Structure which provides us a structure forreportingkeyareaswithinaprojectandalsomeasuringEarnedBusinessValue.AWorkBreakdownStructureprovides“’Adeliverable-orientedgroupingofprojectelementswhichorganizesanddefinesthetotalscopeoftheproject.’[87]
b
ManythinkofGanttchartsandMicrosoftProjectPlanswhentheyhearthetermWorkBreakdownStructure.Thisvisuallyappealingformatallowsanyonetoquicklyseethesalientworkrequiredtoaccomplishtheproject(ProjectataGlance). Thissamplesoftwareproject’sWBSlookslikethefollowing,representingafictitiousATMdevelopmentproject(seeFigure6:FunctionalWorkBreakdownStructure).
Business Value
$-
$100,000
$200,000
$300,000
$400,000
$500,000
$600,000
0 1 2 3 4 5
Sprint
Bus
ines
s Va
lue
ãJeffSutherland1993-2010 160
FIGURE19:FUNCTIONALWORKBREAKDOWNSTRUCTURE
EachofbottomnodesintheFunctionallegisausecase.Ausecaseisnotrequiredbutscenariosofusecasesandstoriesalignwellandhelpproduceusefulsoftware.Usingotherformsofrequirementsdoesnotinvalidatethisstructure.
Notice the numbers in the nodes that represent tangible things that can be valued. The other items arenecessarytodelivertherequiredresultsbutdonothavedirectbusinessvalue.
EARNED BUSINESS VALUE
InordertorepresenttheEarnedBusinessValue(EBV)ofaprojectanditscomponents,anadditiveweightneedstobeassigned. TotalBusinessValueisdeterminedbysomeROIcalculationorequivalent. BusinessValue becomes earned only when the items are done. In Scrum terms thismeans it is an “increment ofpotentiallyshippablefunctionality.”Thus,onlyitemsofdirectbusinessvalue,suchasfunctionalityandtrainingshouldbeassignedweightsotherthanzero.Theotheritemsarethecostofdoingbusiness.Bycallingthem“orphans”theyneedtobeadoptedbyitemsthatdohavevalue(Note:Thisisusefulbecauseitaddressestotalcost,notjustcost-per-featureofaprojectandmakesvisiblethecostofdoingbusinessinsoftware.Also,thesoftwareteamisremindedthedifferencebetweenimportantworkandbusinessvalueoftheoutput).
DetailedcalculationofEBV iswaspublished in theAgile2006proceedings .Here,onlyabriefoverview isprovidedforonecalculation[86].InordertoapplyBusinessValue(BV)toaproject,weneedtocalculatetheBusinessValueIndexes.TheBusinessValueIndex(BVI)oftheentireATMprojectequals1.ForeachlevelintheWBS,theindexis1:ThisisanintermediatevaluethatwillbeusedtocalculatetheBVI.TocalculatetheBVIofsubsequentlevels,youmustalsomultiplytheBVIoftheallnodesabovethelevelyouareconsidering.
Thenextpartisabitmorecomplexbutthepatterniseasyonceyouunderstandit.Remember,theBVIofthenextlevelis1*1=1,theindexoftheProjecttimestheindexofthenextlowerlevel.Becausethefunctionalleghasaweightof3andthebusinessleghasaweightof1,thesumoftheadditiveweightsofthislevelis3+1=4.
ãJeffSutherland1993-2010 161
Thus,theIndexofthefunctionallegis3/4andthebusinessleg1/4:Thiswasobtainedbydividingtheweightofthelegbythesumoftheweightsonthisleg. Thisiswhyadditiveweightsmustbeused. SoinordertocalculatetheBVI,multiplytheindexofthetargetlegbytheindexofallthelegsaboveit.SincetheonlylegabovethefunctionallegistheATMproject,3/4*1=3/4.theweightofthelegbythesumoftheweightsofalllegs.ThustheBVIforthefunctionallegis3/4andforthebusinesslegis1/4.
Usingthesamemethod,theBVIoftheloginusecaseis1*3/4*1*15/43=45/172.IftheTotalBusinessValueoftheprojectis$500,000,thentheEarnedBusinessValue(EBV)realizedbycompletingtheLoginUsecasewouldbe$130,814.
THE UNDERLYING DATA
Managingandreportingeffectivelyisalotofwork.Thevalidityofthereportsisonlyasgoodasthevalidityofthedata. Figure7:WBSinProductBacklogFormatcapturestheWBSandcalculatestheBVIforeachlevel.Notethateachusecasehasbeenbrokenintostories(scenarios)andtheweightsandBVIhavebeencalculatedaswell.Foreachitemmarkedasdone,theEBViscalculated.NoticethebottomrowisdonebutthereisnoEBVbecausethisrepresentsacostofdoingbusiness.
FIGURE20:WBSINPRODUCTBACKLOGFORMAT
So,howdoesthisinformationproduceadashboard?
PUTTING THE DATA INTO THE REPORT
Area Sub-Level Use Cases Value BVI Stories or Features Value BVI Estimate Done Sprint EBV
Product Function Login 15 26.2%Gather Stakeholder Concerns and determine postconditions 0 0% 8 1 1 $ -
Product Function Login Determine Main Success Scenario 0 0% 16 1 1 $ - Product Function Login Code up Main Success Scenario 10 20% 80 1 1 $ 100,626 Product Function Login
Analyze Primary Business Extensions 0 0% 30 1 1 $ -
Product Function Login Code Up "3 Strikes and You're Out" 3 6% 16 1 1 $ 30,188 Product Function Withdraw Cash 10 17.4%
Gather Stakeholder Concerns and determine postconditions 0 0% 12 1 2 $ -
Product Function Withdraw Cash Determine Main Success Scenario 0 0% 20 1 2 $ - Product Function Withdraw Cash Code up Main Success Scenario 7 12% 120 1 2 $ 61,047 Product Function Withdraw Cash
Analyze Primary Business Extensions 0 0% 30 0 2 $ -
Product Function Withdraw Cash Code Up "Quick Cash Options" 3 5% 20 0 2 $ - Product Function Deposit Check 7 12.2%
Gather Stakeholder Concerns and determine postconditions 0 0% 20 1 2 $ -
Product Function Deposit Check Determine Main Success Scenario 0 0% 20 1 2 $ - Product Function Deposit Check Code up Main Success Scenario 10 10% 100 0 2 $ - Product Function Deposit Check
Analyze Primary Business Extensions 0 0% 30 1 2 $ -
Product Function Deposit Check Code Up "Deposit Foreign Currency" 2 2% 50 0 2 $ - Product Function Transfer Funds 10 17.4% 300 0 3 $ - Product Function Buy Stamps 1 1.7% 100 0 3 $ - Product Structure Conversions 0.0% 200 0Product Structure Rewrites 0.0% 300 0Product Structure Refactoring 0.0% 250 0Team Team Management 0.0% 120 0Team Team Team Training 0.0% 40 1 1
ãJeffSutherland1993-2010 162
TheProductBurndown is automatic, using the columns, Estimate,Done and Sprint.Rather thancompleteautomation, theauto-filter feature inExcel isapplied foreachSprint and this can then be easily tabulated into this table (see Figure 7: ProductBurndown).ThisisfoundintheheadersectionoftheProductBacklogintheattachedspreadsheet. Using the subtotal function in Excel provides quick calculations fromfiltereddata. ApplyingnofiltercalculatestheWorkLeftbeforethestartofSprint1.Sprint1wasfoundbyusingacustomfilterwhereSprint“isnotequalto1”.Sprint2wasfoundbyapplyingthefilterwhereSprint“isgreaterthan1.”AslongasanyitemsthataredroppedformanactiveSprintisreflectedintheProductBacklogasplannedforthenextSprint(unlesstheProductOwnerchangesthattoaSprintfurtherout).
CREATING THE PARKING LOT
TheParkingLotiscreatedusingthefiltersaswell.Forthe“DepositCheck”usecase,filteronDepositCheckintheusecasecolumn.Thenumberofstoriesequals5sothiswillgointhatmiddleofthediagram(seeFigure8:DepositCheckProgressIndicator).Thetotalestimatedtimetocompletionis220andthetotalestimateoftimewhere“Done”isfilteredto1is70.Thus,thepercentagecompleteis70/220~32%.Usingtheelementsinthe“ParkingLotConstruction”tabinthespreadsheet,youcancopyandpastetheprogressbar,soitgraphicallyapproximatesthiscompletionvalue.SinceScrumusesforwardlookingestimates,itispossibleforausecaseorfeaturegrouptoshownegativeprogresscomparedtoapreviousreportasnewinformationcausesestimatestoincrease.Thedateisformattedinmonthandyearbutcanbereformattedtoreflectthelengthsofiterations.
FIGURE22:DEPOSITCHECKPROGRESSINDICATOR
EARNED BUSINESS VALUE
EarnedBusinessValueiseasilycalculatedbyfilteringontheDonefieldforavalueof1. ThenfilterontheSprintsuptoandincludingtheonebeingcalculatedusingthecustomfilter“islessthanorequalto.”ThisyieldsthedataintheEBVcolumnforeachSprint(seeFigure9:EarnedBusinessValueData).Thisislocatedinthe“EBV”tabintheaccompanyingspreadsheet.NotethePlannedBusinessValueiscalculatedinitially.Valueis
12%
Deposit Check
DR
( 3 )
( 32% )
March 2005
LEGEND
CompletedAttentionIn ProcessNot Started
Progress Bar
BVI Owner
Feature Group,Use Case Package
or Use Case
# features or stories
% complete
Planned Completion
Sprint Work Left0 23621 20522 1152345
FIGURE21:PRODUCTBURNDOWN
ãJeffSutherland1993-2010 163
typicallyrealizedusingan‘S’curve.Thesampleherehasonly5datapointssothesmoothingfeaturewasusedforthePlannedBusinessValue.Iftheprojectishighlyemergent(littleplanningbeyondthecurrentSprint)theplannedbusinessvaluewillonlybeonerowaheadoftheearnedbusinessvalue.
CONCLUSIONS
BytakingadvantageofthevisibilitythatAgilemethodologiesprovide,wecandelivermeaningfulinformationall the way up to Executive Management using graphicalrepresentations. The addition of a functional work breakdownstructureprovidestheabilitytoviewtheprojectatmultiplelevelsofgranularity.
NOTES
Many other metrics can be derived from the core data, including early warning indicators and schedulevariances.
Howweightsareappliedsotheyareadditivecanbedebatedorcanbesimple.Thisshouldbecoordinatedwiththebusinessandfinancialpeopletousetheircalculatorstohelpderivethis.
Sprint Planned Business Value Earned Business Value0 -$ -$ 1 100,000$ 130,814$ 2 150,000$ 191,861$ 3 300,000$ 4 475,000$ 5 500,000$
Note:This column represents skewed data because of graph smoothing. Because there are only 5 data points, a standard S-shaped Business Value is not represented well without smoothing
FIGURE23:EARNEDBUSINESSVALUEDATA
ãJeffSutherland1993-2010 164
CHAPTER6:SCRUMTUNING
ThereisonlyoneScrumandonesetofcoreprocesses.TheinspectandadaptrulesofScrumallowittobefinetunedforanyorganization.However,oneofthegoalsofthefirstScrumwastomultipleproductivitybyanorderofmagnitude.TheSmalltalkdevelopmentenvironmentcreatedbythefirstScrumatEaselCorporationhadamoneybackguaranteetodoubleproductivitythefirstmonthofuse.Thismaybetheonlymoneybackguaranteeofitstypeseeninthesoftwareindustry.
TherearesomeleadersintheAgilecommunitythatviewproductivityincreasesasmarginal-zeroto25%.TheydoAgiledevelopmenttoimprovequalityandthelifestyleofthedevelopmentteam.Whilethesegoalsarelaudable,theyareaprescriptionforgettingyourdevelopmentteamoutsourced.ThereisaScrumteaminSt.Petersburg,Russia,thatcangiveyouavelocityofseventimesatypicalwaterfallteamanddevelopersalariesaremuch lower than in theU.S.orEurope.Scrumwasdesigned toenable teams in theU.S.andEurope toproducesoftwareathalfthecostofawaterfallteaminIndia,Dubai,Russia,orChina.Keepingajobisthefirststeptojobsatisfaction.
OneofthemotivatingfactorsincreatingScrumwastheBorlandQuattroProprojectdocumentedindetailbyJames Coplienwhen hewas at ATT Bell Labs [2]. Each developer on this project generated 1000 lines ofproductionC++codeeveryweekforthreeyears. Incomparison,productivityontherecentMicrosoftVistaprojectwas1000linesofcodeperdeveloperperyear.TheBorlandQuattroProjectwas52timesasproductiveastheMicrosoftVistaprojectmeasuredbylinesofcode.
MikeCohn,inhisbookonUserStories[69],documentsawellrunningScrumof6peoplethatreimplementedanapplicationwith1/10oftheresourcesofateamthathadusedtraditionalWaterfallpracticestoimplementthatsamefunctionality.ThefirstScrumatEaselCorporationachievedthesamelevelofproductivity.Morerecently,theSirsiDynixprojectdemonstratedthatanoutsourced,distributedScrumonalargeprojectwithover onemillion lines of Java code could achieve about the same high level of productivity [29] as smallcollocatedteams.
ItisthereforesurprisingwhenconsultingwiththemanycompaniesusingScrumthatsomegetlittleincreaseinproductivity,somedoubleproductivity,somequadrupleit,butfewachievetheorderofmagnitudeincreaseforwhichScrumwasdesigned.JustlikeRugbyteams,somecompaniesimplementwellandwinallthetime.Othersdonotdoaswell.Whenyoulookunderthehoodyoufindthattheimplementationofplayisdonepoorlyforsometeams.
Inanattempttoillustrategood,better,andbestimplementationsofScrum,implementationsofScrumwereclassified intoTypeA,TypeB,andTypeC.People thought theseweredifferentScrumswithdifferentcorepractices.Thecorepracticesarethesame.However,thestyleofsomeimplementationsarebetterthanothers
ãJeffSutherland1993-2010 165
with improvedvelocityandqualityachievements.Wewanted toanalyze themso thatScrumpractitionerscouldseetheeffectofrigorouslyimplementingthebasicScrumpracticesanddoingitinwaysthatradicallyimproveddevelopment.ThisisScrumtuning,orwaysofimplementingthecorepracticesofScrumatdifferentlevelsofrigorandaddingsomespecialsaucethattakestheScrumimplementationfrom“good”to“great.”
A better way to describe the evolution of Scrum styles is to show how focus on impediments in Scrumimplementationspaved theway formore rigorous implementationsofScrum. Initiallywewereconcernedaboutmakingateamsuccessful,thenitbecameapparentthemakingtheproductsuccessfulwasahighergoal.Oncethatwasachieved,wenoticedthebestproductdoesn’talwayswin.ThecompanyasawholemustinspectandadaptandbecomeAgilealongwiththesoftwaredevelopers.
ãJeffSutherland1993-2010 166
TYPEBSCRUMCONTINUOUSFLOW:ADVANCINGTHESTATEOFTHEART4
JeffSutherland,Ph.D.
Patientkeeper,Inc.,Brighton,MA,2005.
INTRODUCTION
Scrumisaprocessdesignedtoaddenergy,focus,clarity,andtransparencytosoftwareproductdevelopment.It can increase speedofdevelopment, align individual andcorporateobjectives, createa culturedrivenbyperformance, support shareholder value creation, achieve consistent communication of performance at alllevels,andenhanceindividualdevelopmentandqualityoflife.Scrumaddsvaluetotheworkinglifeofsoftwaredevelopers, while demonstrating to management that radically increased throughput is possible andsustainableovertime.
BACKGROUNDONSCRUM
In1983IbeganworkonbettersoftwaredevelopmentprocessesatMid-ContinentComputerServicesinDenverdoingdataprocessingforover150banksacrosstheUnitedStates.ThisworkwascontinuedatSaddlebrookCorporation,Graphael,Inc.,andObjectDatabasesontheMITcampusinCambridge,MAduring1986-1993.Thegoalwastodeliverbettersoftwaresoonerwhileimprovingtheworkingenvironmentfordevelopers.Softwareteamsareoftenlateandoverbudgetanddevelopersareusuallyblamedforfailure.Theyareoftenpunishedandforcedtoendure“deathmarches”wheretheyregularlyburnoutandquitasqualitydeteriorates,moraledegenerates,andpeopleareforcedtoworkhardertoproduceless.ProfessorPeterSengeofMITviewsthisphenomenonasgeneric toU.S.managementwhose leadership isdedicated tomediocrityanddestroys thespiritofthepeople[88].
In fact, as extensivedata shows froman advanced Scrum implementation at PatientKeeper, Inc. (see laterchapter),lateprojectsareusuallytheresultofmanagementfailure.Resourcesarecontinuallydivertedtonon-criticalactivitiesinsteadofbeingfullyallocatedtoprojects,andcorporatewasteintheformofunnecessarymeetingsandbeaurocracyisrampant.Management’sfailuretotrackandremovethesecorporateimpedimentscausesproductivityloss.ArecentCIOMagazinesurveyshowedthat96%ofCIOsintheUnitedStateshavenoinformationonprojectstatusduringthemiddleofaproject.Flyingblindleadstoregularprojectcrashes.Lack
4FirstpublishedinCutterAgileProjectManagementAdvisoryService:ExecutiveUpdate,2006.
ãJeffSutherland1993-2010 167
ofvisibilityinsideprojects,particularlywaterfallprojects,allowsmanagementtoblamethedevelopersinsteadoftakingtheconsequencesandlearningfromtheiractions.
ProjectsuccessisonlyapreliminaryobjectiveofScrum.Ourgoalistocreateanenvironmentwheredevelopersexceedmanagementexpectationsandcreatenewproductsfasterthanmarketing,sales,andcustomerscanabsorbthem.Thisenablesaworld-classScrumdevelopmentteamthatcanfocusonexceptionallyhighqualityandusabilitywhilepositioningtheircompanytodominateamarket.
Afterevolvingconceptsatfourcompanies,thefirstScrumwaslaunchedatEaselCorporationin1993withthehelpofJeffMcKennaandJohnScumniotales,thefirstScrumMaster.Resultsexceededallexpectationsandtheexperiencewaslife-changingforthedevelopmentteam.In1995,KenSchwaberobservedScruminactionandpresentedthefirstpaperonScrumatOOPSLA.KenandIco-developedScrumforindustry-widedeploymentusingmanycompaniesaslaboratoriestoenhanceandscaleScrum.In1999,MikeBeedlestandardizedScrumasanorganizationalpatternatthePatternLanguagesofProgrammingDesign(PLoP)Conferenceandin2001,the three of us co-authored theManifesto for Agile Software Development alongwith 14 other softwareexperts.TwoScrumbookswerewrittenbyKenSchwaberandMikeBeedle.
Arecentinnovationhasbeenautomatingreal-timedatacollectionforScrumactivitiestoreduceadministrativetimerequiredfromdevelopersandproductmanagers,andtoallowbetterreportingtoseniormanagement.Asideeffectofthisautomationhasbeen(1)collapsingbugreportingandstatusofdevelopmenttasksintoonesystem,(2)realtimereportingonthousandsoftasksondozensofprojects,(3)enhancedabilityfortheProductOwnertosteerthedevelopmentteamintherightdirectionbybetterprioritizationoftheProductBacklog,(4)providing developers automated tasks lists where they can more easily self-manage work, (5) radicallyincreasing throughput formultiple development teams, and (6) reducing project tracking overhead to 60secondsadayfordevelopersand10minutesadayforaprojectmanager.
Asideeffectofreducingdatacollectioncosts tonearzerohasbeen increasedawarenessandmanagementinsightthroughtotaltransparencyofprojectstatus.Thisallowsdevelopmentleadersandseniormanagementtoevaluateprocessbottlenecksandremovewastedeffort.Therelationshipbetweenprojectfailuresandthelack of enforcement of lean manufacturing and lean product development principles can be preciselydemonstratedthroughScrumprojecttrackingdata.LeanprinciplesusedatToyotaandothercompaniesdrivecontinuousqualityimprovementandcanbeusedasamethodtoanalyseprojectmanagementfailures.Rothfuss[89]reviewedPatientKeeper’ssystemandreportedthatitachieved:
• Unprecedentedtransparency
• Companywidevisibility
• Metricsdrivendecisionmaking
• Peerpressureandprideinworkmanshipdrivingproductivity
ãJeffSutherland1993-2010 168
ThesequalitiesmakeScrumusefulforimplementationofLeanpractices.ThesepracticesareusefulforopensourcedevelopmentaswellasISVproductdevelopmentorinternalITinitiatives.Scrumismovingtowardsmorebroadapplication,whilebecomingfaster,better,cheaper,andmorerewardingforcustomers,developers,andmanagers.
IMPROVINGSCRUMIMPLEMENTATIONS
OneofthekeyinfluencesthatledtothecreationoftheAgiledevelopmentprocesses,particularlyScrum,wasapaperonJapanesenewproductdevelopmentbyTakeuchiandNonaka[1].Theauthorspresentedachartshowingproductdevelopmentactivities(requirements,analysis,design,implementation,testing,deployment)separatedintosiloedphasesofwork(TypeAorwaterfall),phasesslightlyoverlapped(TypeB),andallphasesof development overlapping (Type C). The Japanese viewed Type A product development at NASA as anoutmodedrelayracetypeofprocess.TypeBatFuji-XeroxandTypeCatHondatheyenvisionedassimilartoaScrumformation inRugbywhereall teammembers joinedarmsandphasesofproductdevelopmentwereoverlapping.
Figure24:TypeA,B,andCstrategiesfordeliveringproduct[12].
Onecanreinterpret thisdiagramatahigher levelandapply it todifferent typesofScrumexecution.EarlyScrumimplementationsfocusonhelpingthesoftwaredevelopmentteamtobecomemoreproductive.TeamsoftendoworkwithinatimeboxcalledaSprint.IfnoworkisdonewithinthecurrentSprinttoprepareforthenextSprint, timemaybe lostbetween iterationswhilereorganizing for thenextSprint (reset time inLeanmanufacturing).IftheProductOwnerteaminScrumworksonreprioritizing,specifying,andestimating(with
Type A – Isolated cycles of work
Type B – Overlapping iterations
Type C – All at once
ãJeffSutherland1993-2010 169
developers)priortothenextSprintPlanningmeeting,thistimedelaycanbeeliminated.Byreducingresettimetozero,aContinuousFlowScrumallowsworktoflowsmoothlyfromSprinttoSprintandcanoftendoublethroughput.AnAll-at-OnceScrumcanbeenvisionedaspipeliningSprints, i.e.runningmultipleoverlappingSprintsthroughthesameScrumteam.This isthetypeofprocessseenatPatientKeeperandatafewothersoftwaredevelopmentcompanies.
Manycompanies,largeandsmall,areexperiencedwithScrumandhavethechallengeofdeliveringaportfolioofproductsdevelopedbymultiplesynchronizedteams.WhilefewarereadyforaAll-At-OnceScrum,mostwanttoimplementaContinousFlowScrum.Thisinvolvesintroducingleanproductdevelopmentprinciplestoguidetheself-organizationofaScrumimplementationinacompanysetting.Thereisalsovalueinunderstandingandimplementingsomeprinciplesfromleanmanufacturing,leanlogistics,andleansuppliermanagementasleanisasystemofmanagementofanentireorganization,morethanacollectionoftechniquesandpractices.Manyofthesepracticesaredirectlyusefulforsoftwaredevelopment[8].
ENHANCINGSCRUMIMPLEMENTATIONS
The differences in a Team Scrum, Continuous Flow Scrum, and an All-At-Once Scrum have been wellsummarizedbyRyanMartell [90]below.Amajor thrustofScrum isexamining,prioritizing,andremovingimpediments to shorten cycle time and deliver production software faster. Shorter delivery cyclesmeansquickertimetomarket,fasterevolutionoftheproduct,andincreasedknowledgegainedbytheproductteams.Toyotahasdemonstratedthatintroducingleanmanufacturingtoshortencycletimeimprovesproductqualityandoverwhelmscompetition.Newproductisintroducedsofastandfunctionalityincreasesandimprovessoquickly,thecompetitioncannotkeepup.
ãJeffSutherland1993-2010 170
TABLE1:SCRUMCHARACTERISTICS[90].
TeamScrum ContinuousFlowScrum All-At-OnceScrum
Iteration/SprintOverlap
Down-time betweenSprints/No overlap(insufficient ProductOwnerengagement)
Slight overlap(Planning/Prep)
CompleteandMultiple
LevelofInvolvement
DevelopmentTeam ProductManagementTeam EntireOrganization
Release
Every4-6Iterations Every2–4Iterations EveryIteration/Sprint
IterationLengths Fixed Fixed Multiple Overlappingw/varyinglengths
Cycle Time for NewRequestsdelivery
4-6Months 2-3Months Monthly
Releaseperyear Total–2
2Majoryear
patchasneeded
Total–4
QuarterlyMajor(4)
patchasneed
Total-12
Weeklypatch,(8)
Monthlyupdate(8)
QuarterlyMajor(4)
Here,wefocusonContinuousFlowScrumwhichisthenextstepforcompaniesthathavemasteredthebasicScrumprocess.FollowingtheexamplesreviewedbyTakeuchiandNonaka[12]wesetanaudaciousgoal–cutrelease time inhalf,oneof theprimedirectivesof theToyotaPriusproject.Averagedevelopment time fordesignand implementationofanewautomobileatToyotawas fouryears.ThePrius teamdelivered in15months.
Aggressivegoalsputpressureonthesystemandprocessesmustchange.Engineeringpracticesmustimprove.Softwarebuildprocessesmustbeautomated.Testingmustbestartedatthebeginningasitcannotwaituntildevelopmentisdone.Productreportingmustbeautomatedandvisibletoall inthecompany.Alonglistofimpedimentstoprogressemergewhencycletimeiscompressed.InScrum,impedimentsareprioritizedbytheScrumMasterandsystematicallyeliminatedwiththeassistanceofthemanagementteam.
ãJeffSutherland1993-2010 171
InTeamScrum,thefocusisonteamperformance.ContinuousFlowScrumfocusesonProgramManagementperformanceandproductrelease(acombinationofmanagementandProductOwneractivities).ContinuousFlow Scrum does not exist if the company as a whole ignores (or is incapable of resolving) prioritizedimpedimentsraisedbytheScrumMasters.TheProgramManagementteammusttakeresponsibilityfordealingwithtoughissuesembeddedincompanyculturesthatsabotageperformance.
MANAGEMENTCHALLENGESFORACONTINUOUSFLOWSCRUM
TheJapaneseusethetermsMuda,Muri,andMurawhenapplyingleanmanufacturingprinciples.Mudaiswaste.Allwaste,includingeverystepthatdoesnotaddcustomervalue,mustbeeliminated.Muriisoverburdeningpeople,aprocess,orasystem.Slacktimemustbeintroducedintoasystemtoeliminatebottlenecks.Muraisunevennessorunduevariation inprocessorproduct.Process flowmustbesmoothedout.Thesepracticesfollowdirectlyfromqueuingtheory.
Many companies find it difficult to eliminate nonproductive assets, outmoded processes, or unnecessaryhierarchy.Bloatedartifacts,heavyweightprojectplans,andpeoplewhoarenotaddingdirectcustomervalueneed to be moved out of the way. The root of these problems [91] derives from the accounting systemintroducedbySloaninthe1920’stomeasuresuccess.CostAccountingdrivesdeviceoptimizationatthelocallevelwhichsuboptimizesglobalperformance.Higherutilizationofresources(peopleandmachines)becomesan end in itself leading to higher production of inventory, rather than optimizing for sold products.Managementforcespeopletoworkharderandhardertoproducelessandless
The Toyota processminimizes inventory and assumes process change occurs on a daily basis and thesechangesmay affect the entire company. The Director of the Toyota Training Institute points out that thepracticeof“Kaizenmind”assumesthatworkerswillbecomebetteratwhattheydoeveryday[92].Failuretodo so is a corporate crisis and corporate cultures that are impervious to change cannot implement leanmanufacturingpractices.
MuriandMuraarethemostinsidiousanddifficultprocessburdenstoeliminate.Oldstylemanagementtriestoloadupadevelopmentteam110%.Thismakesthemrunslower,justastryingtorunacomputerprocessover80%startstoslowitdown,eventuallycausingthesystemtohangup.ThisMurioverloadcausesMura(disruptionsinflow)bygeneratingprocessbottlenecks.
TheJapanesehavemovedawayfrompushtypeprocessestopulltypeprocesses.Whenaworkerneedstostartthenextstep inaprocess, requirementsandresourcesareprovided just in time.Theworker“pulls” theserequirementsandresourcesonlywhenneeded,allowingworkflowtomoveatasteadypace.Scrumisdesignedto take advantage of lean manufacturing processes and a Continuous Flow Scrum can more than doubleproductivityofTeamScrum.
ãJeffSutherland1993-2010 172
SUMMARY
During2002-2007,theScrumcommunityofpracticeproducedover12000ScrumMastersinanintroductorycourse.While these are only ScrumMasters in training, someof themhave been able to transformentirecompanies.Forexample,Yahoo!hasgonefromoneScrumMasterin2005toover100Scrumteamsin2007.ForeachScrumMaster,thereappeartoover10Scrumteamsactiveonprojects.Thesetensofthousandsofprojects are providing feedback to the Scrum Community through [email protected],biannualScrumGatherings,andanincreasingnumberofresearchandexperiencepublications.
Most of the leading software development companies worldwide have adopted Scrum as a developmentprocesses[93].ManyofthosewhohavesuccessfullyexecutedTeamScrumsforayearormorearereadytomovetoaContinuousFlowScrum.Thisrequiresacorporateculturechangetoeliminateallprocessesthatdonotaddimmediatecustomervalue.Managementhastomovetoafacilitativeleadershipstyleandbreakdowntraditionalhierarchies.
TheToyotaconsultinggrouprecentlyselectedthemostAgilecompanyitcouldfindintheUnitedStates.Insixmonthstheyimprovedproductivityby80%.Scrumhasrepeatedlydemonstrateditcandothesameorbetter.In2006,aCMMILevel5companyintroducedScrumasaprocessimprovementintiative.Defectswerereduced40%,planning costswere reduced80%,andoverallproductivitywasup100%[94].TheynowbidScrumprojectsathalfthecostofwaterfallprojects.WinnersmayputtheircompetitionoutofbusinessjustasToyotaistakingtheleadawayfromGeneralMotors.
POSTSCRIPT
ThispaperprovidedabriefoverviewofaContinuousFlowScrumwhenitwasoriginallypublished.Amoreprofoundinsightintotheseissuescanbefoundthroughstudyofset-basedconcurrentengineering[57]andthetheoryofconstraints[95].
WhenthispaperwasoriginallywrittenfortheCutterAgileProjectManagementAdvisoryService,theeditorsaskedthatitbewatereddown.TheydidnotwantmetostatethatToyotawouldovertakeGeneralMotors.IthadforcefullymadethestatementthatContinousFlowScrumisdesignedtocreatetheToyotaeffectoffourtimes productivity and 12 times better quality than competitors. This level of performance will put thecompetitionoutofbusiness. It is inevitable.Theonlyquestion isoneof timingandoneonlyhasto lookatToyota’simpactonGeneralMotors.LittledidIknowhowquicklythatproblemwouldbecomeofimmediateimportance.
ãJeffSutherland1993-2010 173
GeneralMotorsinrecentyearshasimprovedtheperformanceofemployeessuchthatthetimespentonaddingrealvaluetothecustomerhasmovedfromatraditional20%upclosetoToyota’s80%.Nevertheless,asForbescolumnistMarilynCohenobserves:
“Thedomesticautomakersare inastew,withGeneralMotors inthedeepest.Nevertheless,GM,the largestcompanyinwhatseemstobeadyingU.S.industry,maygetanewlife.Atleastthat’sthepossibilityheldout...(for)analliancebetweenGM’sChiefRichardWagoner,andRenault/NissanMotor’s,CarlosGhosn…Thenlong-sufferingGMbondholderswon’tbeexposedtoaChapter11filing,whichisthefatemanyinvestorsmentallyassignedthecompanynotlongago[96].”
Such is the ToyotaWay side-effect. Of evenmore interest, the strategic target for Toyota and some otherJapanesecarmanufacturersisnotGeneralMotors,itistoeliminatetheinternalcombustionengine.ThelackofAgilityandtheabilitytoinnovateinanenvironmentwhereleadersaretotallychangingtheplayingfieldwillultimatelyforcethecreativedestructionandacquisitionofassetsofthelameandtheweak.
ãJeffSutherland1993-2010 174
FUTUREOFSCRUM:CREATINGASCRUMCOMPANYWITHATYPECALL-AT-ONCESCRUM
JeffSutherland,Ph.D.
ScrumInc.,MIT/CambridgeInnovationCenter2011
“Thisisaveryimportantpaper!Itlaysoutamasterfulseriesofbusinessprocessinnovationsthatdesperatelyneedtobeemulatedbymostorganizations.“TomPoppendieck
ABSTRACT
Scrumwas invented to rapidly drive innovative new product tomarket. Sixmonth releases used to be areasonabletimefromforanenterprisesystem.Nowitisthreemonthsforamajornewrelease,onemonthforupgrades,andoneweekformaintenancereleases.TheScrumdevelopmentprocesswasdesignedtoenhanceproductivityandreducetimetomarketfornewproduct.Inthispaper,oneoftheinventorsofScrumgoesbacktoScrumbasics anddesignsAll-At-OnceScrumusingmultipleoverlappingSprintswithin the sameScrumteams.Thismethodologydelivers increasingapplication functionality tomarketatapace thatoverwhelmscompetitors.TocapturedominantmarketsharerequiresseniormanagementparticipationinaMetaScrumforreleaseplanning,variablelengthSprints,overlappingSprintsforasingleteam,pre-stagingProductBacklog,dailyScrumofScrumsmeetings,andautomationandintegrationofProductBacklogandSprintBacklogwithreal-timereporting.ApracticalexampleofAll-At-OnceScrumdescribeshowmobile/wirelessproductteamsimplementedScrumprocessautomationbeginningin2000andachievedhyperproductiverevenuegrowthbydisplacingdominantvendorsin2007.Administrativeoverheadforover45enterpriseproductreleasesayearislessthan60secondsadayperdeveloperandlessthan10minutesadayforaProjectManager.WhileAll-At-OnceScrumisnotforbeginners,thisprofessionalimplementationofScrumcompanywideisfaster,better,andcoolerthanpreviousimplementations.
1.SCRUMEVOLUTION
Evolutionoccursindynamicresponsetoenvironmentaldemands.NowthattheScrumcommunityhasover100,000 Scrum Masters and hundreds of thousands and perhaps millions of projects under their belt,retrospectioncanhelpguidefutureactivities. Inparticular,whatdidyoudoyesterdaythatworked(Scrumtheory),whatmakessensetodotomorrow(Scrumevolution),andwhatisblockingtheway(Scrumdogma)isworthyofanalysis.
ãJeffSutherland1993-2010 175
OneofthekeyinfluencessparkingthecreationoftheScrumAgiledevelopmentprocesswasaHarvardBusinessReviewpaperonJapanesenewproductdevelopmentbyTakeuchiandNonaka[1].Akeycomponentoftheirpresentation was a chart showing product development separated into silo’s (Type A), phases slightlyoverlapped(TypeB),andallphasesofdevelopmentoverlapping(TypeC).TheJapaneseviewedTypeAproductdevelopmentasanoutmodedrelayracetypeofprocess.TypeBtheythoughtwassimilartoSashimibecauseslicesofworkoverlappedrequiringcollaborationbetweenphases.TypeCtheyenvisionedasScrumwhereallphasesofproductdevelopmentweredonesimultaneously.ScrumisaRugbyformationandtheyviewedan“all-at-once”processassimilartoaRugbyteammovingdownthefieldpassingtheballbackandforthtooneanother.
FIGURE25:TYPEA,B,ANDCSTRATEGIESFORDELIVERINGPRODUCT[12].
AfterdiscussingthenotionofvarioustypesofScrumwithdevelopment teamsatGoogle,Microsoft,Yahoo,Arriba,Adobe,GEHealthcare,andothercompanies,itappearedthatthechartabovecanbeappliedtoahigherlevelofthinkingaboutthreestylesofScrumimplementation.
InearlyScrumimplementations(calledhereaTeamScrum),alldevelopmentoccurswithinthetimeboxofaScrum iteration called a Sprint. A side effect of this approach is downtime between iterations whenreorganizingforthenextSprint.WellexecutedSprintscandoubleproductivityandrepeatedlydeliverprojectsontime,withinbudget,withfunctionalitypreciselytargetedtoend-userdemands.
ByaddingproductdefinitiontasksforthenextSprintintothecurrentSprint,aContinuousFlowScrumallowsworktoflowsmoothlyfromSprinttoSprint.ProductbacklogrequirementsforthenextSprintaredevelopedinthecurrentSprint.WhilethisrequiresthatadevelopmentteamallocateasmallamountoftimefromthecurrentSprinttohelptheProductOwnerestimatetheProductBacklogforsubsequentSprints,itcanenable
Type A – Isolated cycles of work
Type B – Overlapping iterations
Type C – All at once
ãJeffSutherland1993-2010 176
high performance development organizations to deliver more working product than sales, marketing, orcustomerscanabsorb.Byeliminatedthedevelopmentbottleneckthecompanycanadoptnewstrategiesandcreatenewproductsthatwerepreviouslyimpossibletodeliver.MostcompaniestodayhavetoimplementaContinuousFlowScrumoutofnecessitytodeliveracontinuoussetofproductportfolioreleasestothemarket.However,highperformanceofaContinuousFlowScrumrequiresrigorousimplementationofleanpractices.
All-At-OnceScrumcanbeenvisionedaspipeliningSprintsbyrunningmultipleoverlappingSprintsthroughthesamesetofScrumteams.This requiresexperiencedScrumteams,welldesignedproductarchitecture,andautomationofProductandSprintBacklogs.Throughputcanbeenhancedtodeliverdozensofnewreleasesofenterprisesoftwareannually.AMetaScrumis implementedtocreateAgileproductreleasestrategies.Onaweekly basis, companies can alter their direction to deal with competitive threats and market changes.Competitorscanbeoverwhelmedandmarketdominanceachieved.
TakeuchiandNonakaobservedcollapsingphasesofproductdevelopmentimprovedinnovation,throughput,time tomarket, and product acceptance. Asmarket pressures have evolved and changed, it is possible tocollapseScrumSprintstocreatemorebusinessopportunity.MarketdominationisthegoaloftheAll-At-OnceScrum.
2.SCRUMEVOLUTIONINPRACTICE
TheevolutionofScruminfivecompaniesfrom1993-2001hasbeendescribedpreviously[18,40].HerewefocusoncontinuedevolutionofScrumtheoryusingPatientKeeper,Inc.,asatestbed.During2001-2005weautomatedasolutionforaContinuousFlowScrum.ThiseliminatedlosttimeandproductivitybetweenSprintsand, as observedpreviously at Easel Corporation in 1994, significantly increased throughput compared tocompletingworkonlywithintheSprinttimeboxforwhichitisdefined.
Inaddition,wesolvedtheproblemofmultipleprojectspipelinedthroughthesameteam(orsetofteams)andhavebeenrunninganAll-At-OnceScrum forover sixyears.This requiredcarefulautomationof thesprintbacklogwithimprovedtoolsandmetricsinordertomaintainteamfocus.Dailybuildprocessesandautomatedregressiontestingwassignificantlyenhanced.OurapproachtoQualityAssurance(QA)wasmodifiedtoprovideasmallQAteamforeachoffourtosixoverlappingproductionsoftwarereleases.Pairprogrammingwasusedselectivelyandteamprogrammingwascommonwheremanyprogrammersworkedtogetheraroundatablefortheentireday.
TheresulthasbeendeliveryofproductioncodetoanewsetofenterprisecustomersforeverySprintwithmaintenanceSprintsweekly,customerenhancementSprintsmonthly,andnewapplicationreleasesquarterly.By 2004 more than 45 enterprise level releases of PatientKeeper production software were completed,installed, and brought live at customer sites. Many of PatientKeeper’s customers are large multi-hospital
ãJeffSutherland1993-2010 177
systemslikePartners(MassachusettsGeneralandBrighamandWomensHospitals)inBoston,JohnsHopkinsinBaltimore,andDukeUniversityHealthSysteminNorthCarolina.
Recently,PatientKeeperbrokearecordindeploymenttime.ForHCA,thelargesthospitalsystemintheU.S.,PatientKeeperbrought12hospitaldeploymentsintofullproductioninsevenmonthswithnewphysicianandadministrativedesktopandPDAapplications.ThiswasactuallyslowbyPatientKeeperstandards,yetwasahistoricalrecordforHCA.
TheselargeclientsprovideanexcellenttestbedforscalabilityofanAll-At-OnceScrum.Theyrequireahighlevelofproductadoptionbydifficultanddiscriminatingphysicianusers,supportfor largemulti-institutionwirelessnetworks,integrationwithmanyclinicalandfinancialsystemsatsiteswithdiverseITinfrastructures,andthoroughtestingandcertificationbythecustomer.
2.THEFIRSTSCRUMS–TEAMSCRUMANDCONTINUOUSFLOWSCRUM
EarlyTeamScrumswereusefulforeducationandtrainingonthepaceofScrumandparticularlysuitedtonewScrumteams.However,itcreatesalossoftimebetweenSprintswheretheteamisreorganizingforthenextSprintanddidnotadequatelyaddressenterprisescalingissues.
AtEaselCorporationin1993weinitiallyappliedTeamScrumtosoftwaredevelopmentteamswhenwebuiltthefirstobject-orienteddesignandanalysis(OOAD)toolthatincorporatedround-tripengineeringfromdesignto codeandbackagain In a Smalltalkdevelopment environment [22].Codewasgenerated froma graphicdesigntoolandanychangestothecodefromtheSmalltalkintegrateddevelopmentenvironment(IDE)wereimmediatelyreflectedbackintodesign.ThereweresixSprintsforthefirstproductreleaseandthegapbetweenSprintstookatleastaweekandsometimestwoweeks.Asaresult,wecouldonlydo9Sprintsayear,losing25%ofourproductivityascomparedtopotentiallyrunning12Sprintsperyear.ThiswasviewedasaseriousimpedimentbytheProductOwnerandmanagement.
Thislossoftimeneededtoberemovedbecausesurvivalofthecompanydependedondeliveryofaninnovativeproductasearlytomarketaspossible.Eachmonthofdelaycostmillionsofdollarsoflostrevenueandgavethecompetitiontheopportunitytoovertakeus.Forexample,theProductManageratRationalRosewasregularlyshowingusdemosofsupportforroundtripengineeringaswewerestrugglingtobefirsttomarketwiththefirstroundtripengineeringobject-orienteddesigntool.
Inaddition to lossofproductivitybetweenSprints inaTeamScrum, it took timeduring theSprint for thedeveloperstogetenoughclarityabouttheuserrequirementstostartcoding.ItmaybehalfwaythroughaSprintbefore the developers understand the user experiencewell enough to implement a solution. This creates
ãJeffSutherland1993-2010 178
tensionbetweentheProductOwnerandtheScrumTeamconcerninglackofunderstandingofwhattodonext,substantialslippageoffeaturesintosubsequentSprints,anddissatisfactiononthepartoftheProductOwnerwith delays in feature delivery. This churning phenomena can cut Sprint productivity in half, a hugeimpedimentthatneedstoberemedied.
Theneed to startdevelopmentwithadequate functional specificationswasobservedbyMacCormack [97]whenhegatheredextensivedataon29HewlettPackardsoftwareprojectstoassessdevelopmentpractices.One of the strongest productivity enhancers noted in his correlation analysis was completeness of thefunctional specification.WhileAgile specifications are designed to be just enough and nomore, a productspecificationneedstobe“ready”beforeitcanbeallowedintoaSprint.MacCormackshowedthedefinitionof“ready”hasmajorperformanceimplications.
Regarding the use of specifications, there was a significant relationship between the completeness of thefunctional specification andproductivity. Therewas aweak relationship between the completeness of thedetaileddesignspecificationanddefectrate(p=0.078).Theformerresultsuggeststhatdevelopersaremoreproductivetothedegreethatacompletefunctionalspecificationexistspriortocoding.Thisisintuitive,giventhatthefunctionalspecificationoutlinesthefeaturesthatdevelopersmustcomplete.Tothedegreethatthesearestatedupfront,developerscanfocussolelyon“executing”thesefeaturesincode.
Agile developers use a minimum amount of documentation and do not require completeness of thespecification to start a Scrum. McCormack found that completeness of the design specification was notcorrelatedwithenhancedproductivityandonlyslightlyreducedthedefectratewhichisconsistentwithAgilethinking.However,hefoundastrongcorrelationbetweenadequateproductspecificationsandproductivity.ThissuggeststhatminimalfunctionalspecificationsshouldbeclearatthebeginningofaSprintandthatdesignandtechnicalspecificationsarebestdonewithinaSprint.
MacCormack’smultivariateanalysisshowedthreeprimaryfactorsthatlowereddefectrate(earlyprototype,designreviews,andintegrationorregressiontestingatcodecheckin)andtwoprimaryfactorsthatincreasedproductivity (early prototype and daily builds). Releasing a prototype to customers that is only 40%functionally complete increases productivity by 36% and adopting the practice of daily builds increasesproductivityby93%.ThesewereclearlythemosteffectiveAgilepracticesintheHewlettPackardprojects.
IncrementalandearlydeliveryofworkingsoftwareisatthecoreoftheeffectivenessofAgileprocesses. Inaddition,afunctionalspecificationthatiscompleteenoughforthenextinterationtoallowdeveloperstobeginworkwithoutfalsestartswillenhancefeaturedeliverywithinSprintsandimprovethroughput.Despitethefactthattheimplementationphaseisasmallpartoftheoverallcostofasoftwareproject, thebiggestresourcebottleneckonasoftwareproject typicallyoccurswithashortageofexpertdeveloperswhoseskillsarenoteasilytransferable.Constraintanalysisshowsmathematicallythatthebiggestbottlenecksshouldbeeliminatedfirst[95](justasintuningofacomputersystem)andearlydeliveryofafunctionalspecificationforasingleincrementhelpseliminatethecriticaldevelopmentresourcebottleneck.
ãJeffSutherland1993-2010 179
WhileaContinuousFlowScrumcanimproveproductivitywithexperiencedScrumteams,aTeamScrumwithintervalsbetweenSprintstoassuretheProductBacklogis“ready”maybethebestwayforacompanytopilotScrum,eventhoughitmaynotbemostefficient.ItallowssystematicapplicationoftheScrumprocesswithenoughtimetorefineoperationsandregroupbetweenSprints.Italsoforcesall-at-oncetypethinkingwheneverythinghastohappenforaspecificSprintwithinthetimeboxofthatSprint.Initially,thebenefitsintrainingmayoverwhelmthelostproductivity.WithouttheabilitytoexecuteaTeamScrumwell,itisnotpossibletoeffectivelyimplementamoresophisticatedprocess.
ThebenefitsofTeamScrumare:
• Totalfocusoniterationinprocess
• Easeofimplementation
• DevelopingandunderstandingthepaceofScrum
• Clearlydefinediterations
TheproblemswithTeamScrumwere:
• Lossoftimetomarket
• DisruptionofpaceofScrumbecauseofdelaysinunderstandingoftheuserexperience
• Lossofproductivity(andmarketshare)duetoresultingdelays
ScrumhasaprimaryfocusonanimpedimentlistmanagedbytheScrumMaster.Byprioritizingthislistwhichincludespersonal,team,andcompanyissues,theScrumMasterputsanintensefocusonimprovingtheScrumimplementation.WhenremovingimpedimentsmostcompanieswillfindtheyneedtogotoaContinousFlowScrumtomaximizethroughput.
3.CONTINUOUSFLOWSCRUM
Theway toovercome lossof time tomarketwithaTeamScrum is to insert tasks ina currentSprint thatprestageworkforasubsequentSprint.AminimalspecificationoftheuserexperienceforafeatureisdefinedandestimatedpriortotheSprintwhereitisimplemented.ThisallowsSprintstobeexecutedcontinuouslywiththeSprintBacklogalwaysfullatthebeginningofeachSprint.Atthesametime,itrequiresthataScrumTeamallocate resources to help the ProductOwner estimate features for subsequent Sprints during the currentSprint.
ãJeffSutherland1993-2010 180
AcaveatisthatContinuousFlowScrumwillnotworkinacompanythathasnotimplementedasustainabledevelopmentpolicyandprocess.ThatmeansthatScrumteamsdecideonwhattaskscanbeimplementedinaSprintandwhowill implementthemusinganormalworkweekasthestandardwaytodobusiness.ManycompaniesusingScrumstillhavemanagementtryingtojammoreworkintoincrementsthanScrumteamscandeliver in an allotted time. This results in lack of team autonomy, excessive overtime, high defect rates,personnelburnout,andhighemployeeturnover.ThisviolatesafundamentalprincipleofleanproductdevelopandmakesitimpossibleforateamtoenterthehighperformancestateforwhichScrumwasdesigned.
ThekeyindicatorsthatScrumisworkingmustbevisibleinaTeamScrumbeforemovingtoContinousFlowScrum:
• Teamautonomy–theScrumteamis(andfeels)totallyresponsiblefortheirproductandnooutsideagencyimpactstheworkplanoftheteaminsideaSprint.
• TheProductOwnerispartoftheScrumandaffectsproductdesignandimplementationwithinaSprintwithoutdisruptingself-organization.
• Self-transcendence–individualsmovebeyondself-gratificationtofocusonteamperformance.
• Cross-fertilization – expertise is regularly shared across teammembers and no single person is abottleneck.
Fully loadingthedevelopmentqueue inaTypeBScrumatall timeswithoutbuildingasustainablepaceofdevelopmentwillnegativelyimpactmorale.Oncomplexdevelopmentprojects,ittypicallytakesanewengineersixmonthstocomeuptofullproductivity.Ifturnoveris20%,youloseonequarterinhiringanewdevelopmentand two quarters training them. Your development team productivity is down 15% from this alone. Thispersonnelchurncancausedevelopmenttaskstostopandstartasspecializedresourcesmustbeshiftedtocompletethem,reducingproductivitybyanother15%.Ifmoraledrivesthepaceofdevelopmentdownfurther,youmaycutproductivityinhalfwithTypeBSprintsthatareimplementedtooearly.
Conversely,ifTypeASprintsarerunningwell,pre-stagingfunctionalspecificationsintherightwayinaTypeBScrumwilleliminatechurnwithinaSprintanddowntimebetweenSprints.ThishasdoubledproductivityforexperiencedScrumteams.Incompaniesseekingtoexpandmarketshareanddominantamarketsegment,thisadvantageisabsolutelycompelling.
InseveralcompaniesusingScrum,managementandstaffstatedthatmaximizingdevelopmentthroughputwasnotapriority.Thesecompaniesinvariablywerehavingdeliveryproblemsandwereoutsourcingmajorpiecesoftheirdevelopment.Outsourcingwasnotsolvingtheirdeliveryproblemsandinmanycaseswasaggravating
ãJeffSutherland1993-2010 181
it.Inthelongrun,thislasttomarketapproachisalosingstrategy.Ifcompaniesarenotcontinuallygettingbetter,failureisjustamatteroftimeasthecompetitionisalwaysimproving.
3.1 STAGING FUNCTIONAL SPECIFICATIONS FOR A TYPE B SPRINT
ATypeBScrumacceleratesthroughputbykeepingtheSprintbacklogfullandattimes.ThisrequiresprestagingfunctionalspecificationsoftheproductpriortothestartofaSprint.MaintainingtheagilityoftheScrumprocessrequiresaminimalistapproachtofunctionalspecificationswhichisjustenough,andnolessthanjustenough.
Aminimalamountofdocumentationforaproductfeatureistypicallyafewpagesanddefinitelynothundredsof pages. Just enough documentation so that engineers understand the user experience will suffice. Thistypicallymeansscreenshots,datarequirements,workflowfromscreentoscreen,andbusinesslogicthatmustbe executed. Theminimum documentation required to achieve Jacobsen’s overview of an object-orientedanalysisof theproblem[98] isanexcellentguidelineeventhoughitmaybebeyondthecapabilityofsomeproductmanagementorganizations.Fortunately,PatientKeeperhadwelleducatedphysiciansthatservedasProductOwners.Whilesomeofthemhadnoformaltraininginsoftwaredevelopment,theylearnedquicklyhow to elaborate use cases in a way that defined the user experience for the physician when usingPatientKeeperproducts.Inaddition,thesephysicianswhereactionorientedandstronglyresistanttoanalysisparalysis. They avoided time spent on excess documentation, making them excellent Agile developers byinclination.
MovingtoaTypeBScrumrequiresanalysisanddesignresourcesfromthedevelopmentteaminordertohelptheProductOwnercreatefunctionalspecificationsandpre-stagetheSprintbacklogforthenextsprint.Intheworstcase,thismightrequire25%oftheScrumresourcesduringasprint.However,itavoidsthe25%lagtimebetweensprints.Sointheworstcaseyoumaybreakevenonresourceallocation.
TherealgainfromaTypeBScrumishavingtheSprintbacklogfullyloadedatalltimes.Adeveloperneverwonderswhattodonextbecausethequeueisalwaysfull.IftheSprintbacklogisautomated,teammemberssimplylogonatthebeginningofthedayandselfmanagethequeueofworkinfrontofthemonawebpage.TheProductOwnerandScrumMasterarecontinuouslyworkingtoassignitemstoadeveloper’squeueandthedeveloperdecideshowtoordertheworkor,insomecases,willreassignittoamoreappropriateteammember.ThisradicallyincreasesthroughputduringaSprint,oftendoublingit.
Arelevantanalogyiswaterflowthroughagardenhose.Ifthefaucetisturnedonandoff,youdisruptflowandgeneratepressuresurgesintheline.Thesecausesideeffectsandsometimesstructuralbreakdowninwaterlines feeding the hose.Any structural breakdownwill reduceproductivity to zero for an extendedperiod.Keeping the faucet turned on, evenwith reduced flow,may generatemore throughputwithout negativelyimpactingupstreamsystems.
ãJeffSutherland1993-2010 182
AtthetimeofthePatientKeeperfirstmajorroundofventurefundingin2000,IaskedKenSchwabertohelpgetaTypeBScrumlaunchedimmediately.ProductManagementownedtheproductsandtheywererequiredtodefinetheuserexperienceofafeaturebeforeitcouldenteraSprintbacklog.Morespecifically,becauseofthedemandingrequirementsofaphysicianuser,thescreenshots,thelogic,theworkflowbetweenscreens,andthedataitemsrequiredhadtobedefined.Inaddition,aprototypehadtobetestedwithphysicianusersandvalidatedsothatweknewconclusivelythat(1)thefunctionalitywasneeded,and(2)thephysicianswoulduseitwhenitwasimplemented.
The requirement that a Product Owner provide a functional specification sufficient to clarify the userexperiencecreatesapositivedynamictensionbetweentheProductOwnersandtheScrumteams.TheProductOwnercannotgetafeatureintothequeuefordevelopmentunlessitisdefinedenoughfortheScrumteamtobeginworkimmediately,eitherbybuildingatechnicaldesigndocumentorcodingdirectlyfromtheProductManagementspecificationwhenpossible.
Atthesametime,wegavetheProductOwnerresourcesinanySprinttocreateaprototypeofanewfeatureusingveryrapidmockuptoolsthatwouldcreatescreeninteractionsonamobiledevice.Inaddition,theProductOwnerhadcompletecontrolovermovingitemsinandoutoftheSprintBacklogbeforeandduringtheSprint.ThisputstheProductOwnerinthedriver’sseat.ThisiseffectiveonlyiftheProductOwnerknowshowtogettothedestination,i.e.therightproductspecificationisreadyfortheScrumteamtoimplementattherighttime.
ByholdingProjectOwnersresponsiblefordefiningtheuserexperienceforafeaturepriortothestartoftheSprint,arigorousprocesshadtobeintroducedintoProductMarketingatPatientKeeper.TheprocessmadeitcrystalclearthatbuildingnewproductbeganonlywhentheProductManager’sjobwasdone.Atthesametime,itwasimportanttoalwayshavethedevelopmentqueueloaded.Managementinsistedthatthedevelopmentteamnothavedowntime.Thedevelopersself-managedtheirworkqueueandtheworkqueuewasalwaysfulloracompanyemergencywasdeclared,similartothebuildprocessbreaking.TheonlywayProductMarketingcouldgetsomethingon thequeue iswas tocomplete theirminimalist functionalspecification, justenoughinformationinjusttherightformatatjusttherighttime.
ApositivedynamictensionwascreatedbecausetheProductOwner(inthiscase,ProductMarketing)alwayswantsmoreproductinareleaseandtriestojamfeaturesintothedevelopmentqueue.Developersalwayshavemoreworkthantheyhaveavailabletime.InaTypeBScrum,itdoesnotmatterwhetherProductMarketingintroducesnewfeatures intothequeue intimeornot,developmentproductivity isnot impeded.They justworkonwhatisinthequeue.Managementishappybecausetheyarealwaysseeingfeaturesrollout.Ifthemixoffeaturesisoff,theyholdtheProductOwnerresponsiblewhentherequiredfunctionalspecificationswerenotready.
3.2 PRODUCT OWNER AS PART OF THE SCRUM TEAM
ãJeffSutherland1993-2010 183
TheoriginalJapaneseviewofaproductdevelopmentScrumcreatedateamthatwastotallyresponsiblefortheproduct[1].Insomecompanies,suchasIndividualin1996,theProductOwnerwasateveryScrummeeting.Inothers,liketheoriginalScrumsatEaselCorporationin1993-94,theProductOwnerwasontheroadmuchoftheweekandwasalwaysattheFridayScrummeetings[18,40].
TheProductOwnerownsthebusinessplanfortheproduct,thefunctionalspecificationfortheproduct,theproductbacklogfortheproduct,andprioritizationof theproductbacklog.Asamemberof theScrums/heworkssidebysidewiththeScrumMasterto introduceproductbacklogitemsintoaSprintwheretheyarebrokendown into tasksby the team forexecutionasSprintbacklog.AtPatientKeeper, theProductOwnermanagesthemovementoftasksinandoutoftheSprintbackloginconsultationwiththeScrumMaster.
The linkage can be very tight between Product Owner and ScrumMaster with highly skilled people. Forexample,atPatientKeeper,themobiledevicedevelopmentteamleaderistheleaddesignerandcoderontheteam, the Scrum Master, and one of three Product Owners in the company, reporting to both the VP ofMarketing and the Director of Engineering. The two other Product Owners are responsible for clinicalapplications and financial applications on handheld andweb devices. For clinical applications, the clinicalProductOwnerandthemobiledeviceProductOwnerarejoinedatthehipwiththeScrumMasters.Together,they are totally responsible for the business plan, the product specification, andworking day to daywithengineersonproductdesign.
Afterworkingasheadofengineeringinninecompanies,IhavefoundthatthebestwaytothinkaboutScrumresponsibilitiesistothinkofScrumteamasanalogoustoahighperformancecarinarallyrace.TheProductOwneristhenavigatorandtheScrumMasteristhedriver.Theteamistheengine,thechassis,thedrivetrain,and thewheels.TheScrumMaster follows thenavigationaldirectionsof theProductOwnerprecisely anddrivesthecaradroitly.Thecaranditsoccupantsaretotallyresponsibleforwinningtherace.AttheendofeverySprint,otherplayersmoveinandcanmakemodificationstoimprovethetimingofthenextSprint.
NewScrumMasterstendtoviewthisanalogyastooprescriptivetoateamthatassignsitsownresponsibilitiesandself-organizes.Considerafootballteam.Itself-organizesundertheCoachwherethosebestsuitedtoroles,assumepositionsofquarterback,center,tightends,andsoforth.Inthehuddle,theymayquicklymakeminorrefinementstoindividualrolesandresponsibilities.However,whentheballishiked,thereisnodiscussionofwhatanyoneissupposedtodo.Tobesuccessful,theymustknowwhattheyaretodoandexecuteitquicklyandprofessionally.
InsomecompaniesnewtoScrum,engineershaveclaimednooneisresponsiblebecausethereisnota“projectmanager.”Ifyoulookatprojectmanagersintechnicallydrivencompanies,theyareusuallyatthemercyofthetechnicalteam.Asaresult,productmanagement,andthereforeproductownership,isweak.ThiscompromisestheeffectivenessofScrumandpreventsaScrumteamfromenteringahyperproductivestate[26].
ãJeffSutherland1993-2010 184
InawellrunScrum,particularlyaTypeBorCScrum,theScrumMastermustbeabletodrivetheracecarandtheProductOwnermustbeabletodirecttheScrumMastertothedestinationinthetimingnecessarytowinmarketshareandattainprofitability.Failureateitherofthesetasksleadstoreplacementoftheappropriateperson. Successmeans one or both go on to greater responsibilities. For thosewho lead hyperproductiveScrums,careeradvancementisrapidandtheyusuallywindupasCTOs,CEOsoftheirowncompanies,orVPsofEngineeringoflargercompanieswithinafewyears.TheScrumMasteroftheIDXWebTeamlefttheScrumteamtoleadtheU.K.divisionofIDXandclosedover$2Bofnewbusinesswithin3yearsofleavingtheScrum.ThisisaexampleofagreatScrumMasterwholearnedhowtoownthebusinessaswellasthetechnology,sidebysidewiththeProductOwner.
3.3 TYPE B SCRUM CAN ENABLE A HYPERPRODUCTIVE TEAM
GivingtheProductOwnercontroloftheSprintbacklogalongwithstrongaccountabilitybuildsstrongproductmanagers.Italsoconditionsthedevelopmentteamtomoverapidlytowardsthegoalwithoutanalysisparalysis.Acombinationofdynamicforceswinsaracewhenaforcefuldriveriscoupledtoahighperformancesportscarorahighspiritedstallion.Thesamephenomenonhappensonsportsteamswheneveryoneunderstandstheplaysandcanexecutethemimmediatelyondemand.Itallowstheteamtomoveuptoahigherlevelofplaywherethebasicmovesareonautopilotandsuperlativemovesarepossible.ThefirstScrumbeganexecutingTypeBScrumastheymasteredtheprocess.Theywereabletoenterthe“zone”usingthistechnique,wheretheycoulddeliverfunctionalityfasterthanthecustomers,themarketingteam,orsalescouldabsorbproduct.The feeling of power in a development team that can deliver more product than anyone can absorb isexhilarating.
Scrumwasdesignedforthishyperproductivestate,togetordinarydeveloperstofunctionasachampionteam.Itonlyhappenstoabout10%ofScrumsanditonlystartstohappenwhentheorganizationmovestoaTypeBScrum.Thedoublingofthroughputfromateamthatisalreadyveryproductiveresultsinanorganizationalbreakthrough.
4.TYPECSCRUM
Scrum is an organizational pattern that is designed for an activity that is difficult to control because itspredictabilityislimited[26].Itisusefulinanycontextwheretheactivityrequiresconstantchangeindirection,unforeseeninteractionwithmanyparticipants,andtheneedtoaddnewtasksasworkunfolds.ThesefactorswereamplifiedatPatientKeeperwhenitreceiveda$50Mroundofventurefundingin2000.
Adecisionwasmadetobecomeaplatformaswellasapplicationcompanywithasoftwareframeworkandopenapplicationprogramminginterfaces(APIs)thatwouldallowintegrationwithdevelopmentpartnersonbothbackendserversandthefront-enddevices.Awebservicesplatformarchitecturewasselected.
ãJeffSutherland1993-2010 185
FIGURE26:PATIENTKEEPERPLATFORMARCHITECTURE[99].
Inadditiontotheserver/networkarchitecturebasedonJavaandXML,acrossplatformsoftwareframeworkonPalmandPocketPChandhelddeviceswasimplementedinC/C++.ThisframeworkprovidedopenAPIsandasoftwaredevelopmentkitthatallowedthirdpartyvendorsandenduserstotightlyintegratetheirmobileapplicationswithotherapplicationsalreadyavailableonahandhelddevice.
ThetightintegrationbetweensoftwarecomponentsrequiredtightintegrationofsoftwaredevelopmentteamsinternallyatPatientKeeperandexternallywithpartnersandoffshoredevelopers.This,combinedwithtimetomarket pressure and rapid growth of new client deployments in large enterprises, each demanding newincrementsoffunctionalityalongwith30-90dayinstallations,forcedanewtypeofScrumtobeimplementedatPatientKeeper.
4.1 CASE STUDY CONTEXT
PatientKeeperbuildsasoftwareplatformthattakesinformationfrommanyclinicalsystemsacrossmultiplehospitalsandclinicsandpresentsitonanintuitiveuserinterfacetophysiciansusinghandhelddevicesandtheweb.Theapplicationsoftwarehasafourtierarchitecturewithfourlevelsofdatacache:
We Make Healthcare MobileTM
Custom CDR CernerMeditech…
DepartmentalSystems
InterfaceEngine
Internal Web Services
Adapter Adapter Adapter
Middle tierDatabase
Process Manager
Cache
PDA – Web Portal - External Web Services - Telemedicine
MER
Web ServicesBus
Architecture
Workflow/LogisticsManagement
ExecutionTier
RulesEngine
Cache
WorkflowEngine
Cache
LogisticsEngine
Cache
Web Server
Application Server
ãJeffSutherland1993-2010 186
• Primarydatasourceisaclinicaldatarepository
• Dataisforwardcachedinamini-clinicaldatarepository
• In-memorycacheexistsonamiddle-wareservertoimproveperformance
• Onahandhelddevice,thedataisstoredlocally
Softwareanddatamustbeconsistentacrossfourtiersatalltimes.ThisforcedPatientKeepertogotototallyintegratedbuildsatalltimestoassurethatsoftwareinallfourtiersofthearchitectureworkedconsistently.Softwaretestinghastovalidatethatversionofallarchitecturallayersworktogethertoprovideconsistentdatatotheenduser.
Thedeveloperteamworkingonthisproductwassplitintoabackendintegrationteam,aclinicalrepositoryteam,amiddlewareserverteam,twoPDAteams(PalmandPocketPC)andaWebteam.Itwasnecessarytotightlycoupletheseteamstogethertoassurethatallsoftwarewasinteroperableallthetime.
4.2 CASE STUDY MARKET REQUIREMENTS
As an early-stage, venture funded company, PatientKeeper had to establish a new product offering in therapidlygrowingmobile/wirelessmarket.Earlycustomershadtobeimplementedasquicklyaspossiblewithlimited functionality. Subsequent customers needed to be installed as soon as possible with upgradedfunctionality.Theimperativewastogainmarketshareandachievemarketdominanceinahighlycompetitiveenvironment. Speed to market needed to be used as a strategic weapon. Company viability and successdemandedit.
The customer base rapidly evolved to 5-10 hospital systems to be installed each quarter. Each group ofhospitals neededmore extensive functionality in a rapidly growing portfolio of applications that includedbusinesspartnerswithintegratedbackendclinicalsystems,portalvendors,andhandhelddeviceapplicationvendors.Amajorreleasewithnewapplicationswasrequiredonaquarterlybasis.
Customersconsistentoflargehospitalsystemswithmultipleinstallations,academicresearchinstitutionswithintegration into home-grown healthcare research and treatment systems, and medium size communityhospitalswith different requirements. The quarterlymajor release had to be supplementedwithmonthlyminorreleasestoallowmultiplenewinstallsofsimilarclientstotakeplaceonamonthlybasis.Finally,bugsandunanticipatedimplementationissuesthathadtoberesolvedtogoliveatnewsitesrequiredmaintenancereleaseseveryweekortwo.
ãJeffSutherland1993-2010 187
4.3 CASE STUDY PROBLEM
Thechallenge forPatientKeeperquicklybecamehowtosimultaneouslydoweekly,monthly,andquarterlyreleasesofasystemthatwastightlyintegratedacrossfourarchitecturallayerswithsixsoftwaredevelopmentteamsthatneededtobetightlycoupledtoassuredataandapplicationconsistencyacrossmultiplebackends,diversewirelessnetworks,andmultiple frontenddevices.Furthermore,eachreleasehad tobe testedandcertifiedacrossalllevelsofarchitectureandapplicationsfordeployment.
PatientKeeperstartedupasaScrumcompanydoingdailymeetings.TypeBScrumhadbeenimplementedfromthebeginningwithstrongproductownersrequiredtoproducefunctionalspecificationsbeforeanyproductbacklogitemcouldbetransformedintotasksforasprintbacklogtobeimplementedinthenextiteration.ThechallengewastoaautomateprojectmanagementtomonitorthousandsoftasksacrossdozensofreleasesayearwithoutdisruptingtheScrumprocess.
4.4 CASE STUDY FORCES Resourceconstraintsforcedeverydevelopertobefocused100%onbuildingthesystem.ScrumMastersandteamleadersspentthemajorityoftheirtimedesigningandcodingthesystem.Separateprojectleaderswerenotanoption.
Highcaliberdevelopers,manywithdoctoraldegrees,didnotwantexcessiveadministrativeoverhead.Theyfelt that project management could be automated and taken to a new level of efficiency. The CTO ofPatientKeeperwasaskedbytheScrumteamstoorganizeaprojectmanagementsystemthatrequiredlessthan60secondsperdayofadministrativetimeperdeveloperandlessthan10minutesperdayforaScrumMastertoprovidecomprehensivereportingtomanagement,thedevelopmentteam,andotherareasofthecompany.
• Estimationwasimportant.Howweredevelopersgoingtoprovidevalidestimatesandupdatetheminlessthansixtysecondsaday?
• Planning and prioritizing takes time. How was this going to be accomplished without impedingdevelopmentthroughput?
• Architecturewascriticalforaplatformcompany.HowwasitgoingtoevolveusingtheScrumprocesstoprovideflexibility,scalability,performance,reliability,andmaintainability?
• Customerrequirementsintheformofusecasesthatcouldberapidlytransformedintodeliverablecodewereessential.Whowasgoingtodothemandhowwouldtheybedelivered?
4.5 TYPE C SCRUM SOLUTION TheTypeCScrumsolutionrequiredseveralinnovationsthataffectedallpartsofthecompany.Ineffect,thecompanyhadtobecomeaScrumcompanywithallactivitiesdrivenbyanautomateddatasystemthatreflectedreleaseplanningandSprintexecution,aswellasinstallationandsupportteamandcustomerfeedback.
• Teamorganizationwaschanged
• Buildprocessbecamemorehighlyautomated
• Regressiontestingautomationimprovedsignificantly
ãJeffSutherland1993-2010 188
• Alldatacollectionhadtobeautomated
• Newtoolsfordatacollectionandreportedhadtobedeveloped
• AMetaScrumhadtobecreatedtoallowcompanyleadershiptomanagemultiplesimultaneousproductreleases
• Newreportshadtobedeveloped
• Thecompanyhadtobecometotallytransparent.Alldatawasavailabletoeveryoneinrealtimeallthetime.
4.5.1TEAMORGANIZATION
FIGURE27:OPENSPACEFORTYPECSCRUMOFSCRUMSDAILYMEETING
The daily Scrum meeting quickly evolved into a daily Scrum of Scrum meetings. All members of thedevelopmentteamarepresentfor15minutesmeetings.Teamleadersdomostofthereporting:
• TheScrumofScrumsisareleaseteam.ThispatternwasoriginallyestablishedatIDXSystemsin1996[100].TheDailyScaledScrummeetingisfocusedonthemultiplereleasestobedeliveredinsidethesprint.
• Whatdideachofthesixintegratedteamscompleteinthelast24hours?TheScrumofScrumsleaderlogswhatstorieswerecompleted,thestatusofeachreleasetobeshippedthatweek,andsendsoutanemailtothecompanyimmediatelyfollowingtheDailyScaledScrum.
• Blocksfoundinperformingstoriesinthelast24hoursarelogged,reported,andfollowed-upafterthemeeting.Anythingthatcanbesolvedinsixtysecondsisresolvedinthemeeting.
ãJeffSutherland1993-2010 189
FIGURE 28: SUMMARY OF DAILY EMAIL AFTER SCRUM OF SCRUMS MEETING SHOWS SEVEN RELEASES IN PROGRESSSIMULTANEOUSLY.ALLTEAMSWORKONALLRELEASESANDALLRELEASESRESULTINCUSTOMERDEPLOYMENT.
TheDailyScaledScrumtakesplaceatthesametimeandplaceeveryday.Anopenspacewassecuredbythedevelopmentteamforthispurpose.Pairprogrammingwasdoneprimarilyontaskswithdifficultdesignandcodingrequirements.Manyof thedevelopersstayed in theopenmeetingspace for theentiredayworkingtogetherasagroup.Innovativeandopencubespaceandafewofficesandconferenceroomsareprovidedforthosewhoneedquiet,focusedtime.
TherapidpaceofdeliveryofproductioncodereleasesinitiallycreatedaQualityAssurance(QA)bottleneck.ThesolutionwastoassignansmallQAteamtoeveryrelease.QAwasexpandedtofoursmallteamsof2-4people.Thisenabledthemtoworkcontinuouslyonfourofthetoppriorityreleases.IntheFigureabove,wheresixreleasesarebeingsimultaneouslydeveloped,QAisdoingfinalreleasetestingandpackagingonfourofthem.QAispartoftheScrumofScrumsandreportsondailystatusofongoingreleases.
4.5.2DATACOLLECTIONAusergroupstudyandfocusgroupanalysiswasperformedfordatacollectionfortasks,estimates,andupdatesthatwouldbeusedtoautomatethestandardScrumburndowncharts[18].AwidevarietyofScrumtrackingtools had been used bymembers of the team in various companies over a 15 year period, none of themconsideredadequate.The60secondrequirementfordataentryimpliedthatanewapplicationwouldnotbepossible,becausesimplystartingupanewapplicationmightrequire60seconds.
Thebestapplicationtousewasonethatdevelopershadtouseeveryday,thebugtrackingsystem.Inaddition,thespeedatwhichdeveloperscoulddodataentrywasdependentonthequestionstheywereasked,andtheorderinwhichtheywereasked.Itwasdeterminedthatonlythreequestionswouldbeaskedasdeveloperscouldanswerthemwithoutthinking,theycouldgiveagutlevelresponse:
• Whatistheinitialestimateforthistaskifitisanewtask?
© Jeff Sutherland and ADM 2004
Typical Day in a Type C ScrumScrum Master email at close of Scrum daily meetingFriday Releases 19 Nov 2004n 245g5
– getting feedback from Cerner, – they're trying to get micro susceptibilities data into the test system– added MAR suppression to address issue at SOM
n 245m– upgrade testing this morning, should release by noon
n 246– 246g1 palm released with timeout issue fixed– 246i - post t-giving
n 251b2– SUNY patched released last night / installed into SUNY test system
n 251d– Mt Sinai release, should release by noon
n 251e– Monaco clinicals, targeting Alverno
n 3.0.1 102 open PRs, 57 verification (down from 110 on Monday!)– beta release today
ãJeffSutherland1993-2010 190
• Atthismoment,howmuchtimehaveyouspentonthistask?
• Atthismoment,whatpercentcompleteisthistask?
Theseweretheonlyadditionaldataitemstobecollecteddailyfromdevelopersfortasks.Allotherdataanalysisandreportingwastobeautomated.
4.5.3TOOLSFORDATACOLLECTIONANDREPORTINGPatientKeeperusedtheopensourceGNATSbugtrackingsystem[101].Sincedevelopersneededtousethebugtrackingsystemdaily,therewasnoadditionaltimeoverheadforopeningtheapplicationtoentertaskdata.
GNUGNATS isasetof tools for trackingbugsreportedbyusers toacentral site. Itallowsproblemreportmanagement and communication with users via various means. GNATS stores all the information aboutproblemreportsinitsdatabasesandprovidestoolsforquerying,editing,andmaintenanceofthedatabases.
Thankstoitsarchitecture,GNATSisnotboundtoasingleuserinterface–itcanbeusedviacommandline,e-mail,Emacs,oranetworkdaemon,butisusuallyusedwithaWebinterface.TogetherwiththefactthatallGNATSdatabases and configuration canbe stored in plain text files, it allows easyuse andprovides goodflexibility.Basically,iftheGNATStoolsdonotprovideeverythingyouneed,youcanaddyourownadditionalutilitiesusingstandardGNUtools.http://www.gnu.org/software/gnats/
APERLexpertonthedevelopmentteamwasassignedtobuildutilitiesaroundGNATStosupportScrum.Thesewereadditionofrequireddata items,newqueries,minorchangestotheuser interface,andautomatedfiledumpsformanagementreportingviaExcel.SampledataitemsmaintainedbyGNATSareshowninFigure3below.
ãJeffSutherland1993-2010 191
FIGURE29:TYPICALDATAITEMSINGNATSFORPROBLEMREPORTINGBYEMAILORTHEWEB.
It was decided that sprint tasks would be treated like problem reports. This minimized new data entryrequirementsandallowtasksandbugstobepackagedtogetherseamlesslyforarelease.OnlythreedataitemswereaddedtoGNATSfordeveloperentry:
• Initialestimate
• Daysinvested
• %complete
Theinitialestimatewasfixedatinitialentryandcouldneverbechangedinordertoallowforaccuratehistoricalreporting of estimates versus actual time to complete tasks. Two additional data items were added forreportingpurposes.Theseareautomaticallycalculatedfromthethreeitemsabove.
• Daysremaining
• Actualtimetocomplete
ãJeffSutherland1993-2010 192
Iftheinitialestimateis2days,forexample,andnoworkhasbeenaccomplished,thedaysremainingare2days.Ifadeveloperhasinvested1dayandstatesthatitis25%complete,GNATScalculatedthedaysremainingas3days.Initialestimatesareautomaticallyexpandedbasedonrealtimedata.
FIGURE 30: DEVELOPERWORKSTATION TASK LISTING FOR ASSIGNED TASKS FOR A SPRINT. RIGHT CLICK ON THEMOUSEGENERATESDROPDOWNLISTTHATALLOWSANYDATAITEMTOBEUPDATEDALMOSTINSTANTANEOUSLY.
ThecumulativetimeremainingforareleasecanbeobtainedinrealtimebyanyoneinthecompanywithaccesstoGNATS.AtPatientKeeper,thatiseverypersoninthecompany.Thedaysremainingforalltasksassignedtoareleasearetotaledtocalculatecumulativebacklog,thenumberchartedonaScrumburndownchart.Becausetherearethousandsoftasksinthesystemandanytaskedthatistouchedisupdatedeverydayitistouched,thephenomenonofstatisticalregressiontowardsthemean[102]makesthesummarydataoncumulativetimetoreleaseveryaccurate.Itachievestheholygrailofaccountingsoftware,microcostingofeveryactivityinacompany[103].
4.5.4PRODUCTDEVELOPMENTPROCESSREFINEMENTSProductManagementservesastheProductOwneratPatientKeeperandmustprovidefunctionalspecificationsforfeaturesthatsufficientlydescribetheuserexperiencesothatdeveloperscanbegindesignandcoding.Thistypicalmeansscreensshots,workflowbetweenscreens,businesslogic,anddataitemsrequired.Aworkingdemo is normally prototyped and approvedby a physician user groupbefore a feature can enter a sprintbacklog.
TheProductOwnercontrolstheGNATSentriesforareleaseandthebugflowintothesystem.BugsinitiallygointoatriagecategoryandtheProductOwnerassignsthemtoareleasebasedonpriority,customerrequests,implementation issues,andso forth.Featuresare initiallyplaced intoGNATSasplaceholdersassignedtoarelease.Developerscanpickthemupandtranslatethemintotasksspecifictoasprint.
ãJeffSutherland1993-2010 193
Figure7showsaburndownchartgeneratedbyGNATSforPatientKeeperRelease6.ItshowsproductbacklogaccumulatingasplaceholdersuntiltheSprintstartedon12June2002.Atthattime,developersbeganbreakingdownproductfeaturesintoestimatedtasksforthenextsprint.Thisdrovethebacklogupasnewtasksarediscoveredthatwereunanticipatedinoriginalproductfeatureestimates.
On26June2002,thePatientKeeperCEOdecidedtocommitthecompanytoamajoruser interfacereworkduringRelease6.WhentheProductOwnerenteredover80newtasksintothereleasetheburndownchartshotupquicklyinatwodayperiod.Thiswasvisibleimmediatelytotheentirecompanybecauseburndownchartsweremailedouttothecompanyonadailybasis.
Thiscausedalotofconsternationbecausetherewasnowaythatthisiterationwouldbeastandard30daySprint.PatientKeeperhasaweeklyMetaScrum[100]meetingwhichincludesleadersfromalldepartmentsinthecompanywherereleaseprioritiesandfeaturesarereexaminedonaweeklybasis.Itwasdeterminedthatthe value to the company of refining the user interface of the product was very high in a competitiveenvironmentandthesprintwouldbeextendedto24AugustbasedontheScrumburndownchart.ThiswouldrequirethedevelopmentteamtohaveperfectdevelopmentdaysfromthebeginningofJulythroughtheendofAugust.
Anidealdeveloperdayistheamountofworkthatcanbeaccomplishedifadeveloperworksuninterruptedlyfor a normal work day. Most teams preplan to get 40-60% of a development day due to meetings andextraneousinterruptionsforteammembers.Getting180developerdayswithan8personteamin42calendardayswithoutovertimewasnotgoingtobeeasy.ThepolicyatPatientKeeperwastosustainabledevelopmentusinganormalworkweekwithnightandweekendactivityrequiredonlywhenrareemergenciesoccurred,i.e.productioncustomersharddowninthefield.
PatientKeeper had the advantage of smooth running Scrums with few interruptions other than releasepriorities.Asaresult,theirnormalvelocityornumberofdaysworkaccomplishedperdayperdeveloperranbetterthan60%.Yearsofhistoricaldataalsoshowedtheyfinishedtheirtasks inanaverageof85%oftheoriginalestimate.Thisoftendidnotshowupincalendardaysduetoreleasepriorityconflicts.Howevertheleadership team realized that slippage was normally due to management prioritization problems, notdeveloperslippage,andGNATShasmountainsofdatatoproveit.
Thesolutionwastofocustheentireteamononereleaseandovera42calendardayperiod,or30businessdays, thedevelopersdelivered180daysofworkforavelocityof2daysofworkforevery3businessdaysinvested or 66% of the ideal. This was planned in advance, the MetaScrum leadership repositioned allcustomersatthebeginningofJuly,andovertimewasnotrequired.
ãJeffSutherland1993-2010 194
FIGURE31:SCRUMBURNDOWNCHARTAUTOGENERATEDBYGNATS.PRODUCTMANAGEMENTWASENTERINGPLACEHOLDERSFROM PRODUCT BACKLOG UNTIL 6/12/2002WHEN THE SPRINT STARTED. IT EXTENDED BEYOND 30 DAYS FOR REASONSDESCRIBEDELSEWHERE.THEPAINOFTHISSCRUMRECONVINCEDEVERYONETHAT30DAYSISTHEMAXIMUMSCRUMLENGTH.
The capability of total transparencywhere all data is available to everyone extends the concept of globalvisibility during a Scrum to the entire company. This allows all parts of the company to replan activitiesroutinely. In the case of PatientKeeper, new Sprints can be started, changed, or destroyed in the weeklyMetaScrumwithoutdisruptingthefocusoftheScrumteams.
4.5.5PROJECTREPORTINGTheGNATSreportingsystemwasrefinedtoallowsophisticatedmanagementreporting.Twoofthemostusefulexamplesare(1)trackingthequalityofaproductportfolioand(2)automatedgenerationofGanttchartsforMetaScrumplanning.
4.5.5.1TrackingQualityofaProductPortfolio
Ausefulmeasureofproductquality,codestability,andforwardprogressisachartthatshowsarrivalofnewdevelopmenttasks,completionofdevelopmenttasksthatchangestatustoverification(wheretheybecometheresponsibilityoftheQAteam),andclosingoftaskswhentestingbytheQAteamiscomplete.Thecumulativenumberofoutstandingdefectshasbeendividedby10inFigure8toallowchartingofthecumulativetotalinthesamerangeanddailydefectarrival.
ãJeffSutherland1993-2010 195
FIGURE32:DAILYARRIVALOFDEFECTSALONGWITHCUMULATIVEDEFECTCOUNT.COMPANYOBJECTIVEISTODRIVETOTALBUGCOUNTACROSSADEPLOYEDPORTFOLIOOFRELEASESBELOW100. THIS IS CHALLENGINGASTHEREAREABOUT100HOSPITALSDEPLOYEDONTHE2.4.*RELEASES.
4.5.5.2GanttChartforMetaScrumPlanning
Ganttchartsareusefulforplanningpurposes.However,theyarepoorfortrackingsoftwareprojectsbecausedependencieschangeonadailybasis.AfulltimedevelopercanbeabsorbedkeepingMicrosoftProjectuptodateforasingleScrumteamandScrumwasdesignedtoeliminatethiswastefulactivity.Forthelastsixyears,PatientKeeperhasevaluatedwhethertohaveanyprojectmanagementotherthanProductOwnersandScrumMasters.Thedecisionhasalwaysbeenthattheyareunnecessarywaste.
AMetaScrumcanfindaGanttchartuseful,butonlyifitismachinegenerated.AhumangeneratedGanttchartisinaccurateinthebeginningandcompletelyoutdatedwithinaweekinafast-pacedcompany.AneffectiveGanttchartcanbecalculatedinrealtimebasedondatacaptureinthereportingsystem.
© Jeff Sutherland and ADM 2004
Defects Open/Closed by Day:Managing Quality of Product Portfolio
ãJeffSutherland1993-2010 196
FIGURE33:DYNAMICALLYGENERATEDGANTTCHART.ENDPOINTSREFLECTANTICIPATEDRELEASEDATEINDAYSFROMDAYOFGENERATION.
DynamicGanttchartscanbegeneratedbystaffmembersforasmallteam,orbyteamsforacompany.ThechartaboveshowstworeleasesbrokenintodevelopmenttasksandQAtasks.TheXaxisisdaysfromgenerationday.TheYaxisisstaffmembersbynameshowingassignmentsfortheteam.
TheresultofanautomaticallygeneratedGanttchartisasurprisetotraditionalmanagers.Itwillshowmostindividualsareloadedonly1-3daysout.Theywillchoosetheirnexttaskwhentheycompletetheircurrenttask.KeypeopleliketheleadarchitectorQAmanagerwillhavetasksqueueduptobepassedofftowhoeverisreadytotakethem“justintime.”
WhenPatientKeepermanagerswereasked if theywanted tomanage resources toallowanautogeneratedGanttchargetoshowreleasedatestheywerecountingon,theyimmediatelydeclined,notingthatdisruptionof a self-organizing systemwould radically cut the velocity of the team and create unnecessarywork formanagers.TheygaveupthenotionoftryingtouseaGanttcharttoplanreleasesandwentbacktotheProductOwner’sroadmapforreleaseplanning.ThisisamilestonebasedtimelinethatshowstheProductOwner’sbestestimateofreleasedateswithspecifiedhighestvaluesfeaturesbasedonknownteamvelocitiesandprojectdependencies.
4.6 TYPE C SCRUM RATIONALE AsnotedinourPatternLanguagesofProgramDesignpaper[26],“itisveryeasytoover-orunder-estimate,whichleadseithertoidledevelopertimeortodelaysinthecompletionofanassignment.Therefore,itisbettertofrequentlysamplethestatusofsmallassignments.ProcesseswithahighdegreeofunpredictabilitycannotusetraditionalprojectplanningtechniquessuchasGanttorPERTchartsonly,becausetherateofchangeofwhatisbeinganalyzed,accomplished,orcreatedistoohigh.Instead,constantreprioritizationoftasksoffersan adaptivemechanism that provides sampling of systemic knowledge over short periods of time. Scrummeetingshelpalsointhecreationofananticipatingculture[104]becausetheyencourageproductivevalues:
© Jeff Sutherland and ADM 2004
Dynamic GANTT Chart:Managing Mult iple Releases
0 5 10 15 20 25 30 35 40
abonventre
ctashjian
dcohan
dkoulomzin
gaspeslagh
jcarp
legli
rkozak
sally
sanj
svakil
Day
s
Release 2.5.1c & 3.0
R2.5.1cVer 2.5.1cR3.0Ver 3.0
ãJeffSutherland1993-2010 197
• Theyincreasetheoverallsenseofurgency.
• Theypromotethesharingofknowledge.
• Theyencouragedensecommunications.
• Theyfacilitatehonestyamongdeveloperssinceeveryonehastogiveadailystatus.
InaTypeCScrum,theurgency,sharing,communications,andhonestybehaviorsareextendedcompanywide.“From the Complexity Theory perspective [105, 106], Scrum allows flocking by forcing a faster agentinteraction,thereforeacceleratingtheprocessofself-organizationbecauseitshiftsresourcesopportunisticallythroughthedailyScrummeetings.[26]”Whenextendingcompanywide,theentirecompanycanself-organizeonaweeklybasis.Thefollowingbehaviorsbecomecommonplace:
• There is never an unexpected late release as problems are seen long before the release date. Thecompanyself-organizesaroundtheissuesraisedintheMetaScrum.
• ChangesincustomerrequirementsarereflectedimmediatelyinproductbacklogandrelevantSprintbacklog.DecisionsaremadetoreorganizeonaweeklybasisintheMetaScrum.
• Company imperatives andmanagement changes that affect product backlog aremade only in theMetaScrum.Thiseliminatesmostpolitics,lobbying,andcloseddoormeetings.
• Customer impactandschedule impactsaredealwith immediately in theMetaScrumat thetimeofdecision.TheCEO,salesstaff,andaccountmanagementwalkoutofthemeetingwithassignedtaskstodealwithcustomersaffectedbydecisions.
4.7 TYPE C SCRUM RESULTING CONTEXT ThemovetoaTypeCScrumtoimprovedevelopmentproductivityhadfarreachingeffectsonthecompanymakingitmoreflexible,moredecisive,moreadaptable,andabetterplacetowork.ThesameeffectscommonlyseenonScrumteamswerereflectedthroughoutthecompany.
Projectmanagementwastotallyautomated.Theresultispaperlessprojectmanagementandreporting,largelywithouthumanintervention.Scrumexecutionhasbecomeexceptionallyefficientandtheautomatedtrackingsystemhasbecomemissioncritical.Burndownchartshaveevolvedtoframetheentirestatusofaprojectononechart.ThechartbelowinstantaneouslyreflectsprojectstateforRelease3.20ataglancetothosefamiliarwiththedata.Withalltasks
entered at 16 hours or less and bug fixes typically less than a day, the aggregate number of tasks can bemonitoredanddownwardvelocityishighlypredictiveofdeliverydate.Informationispresentedasfollows:
• DarkBlueDiamond–Release3.20currentopen–cumulativeworkremaining
• YellowTriangle–Release3.20dailyclosed-itemsclosedbyQAeachday
• PurpleStar–Release3.20totalclosed-cumulativeclosed(onscaleatright)
• PinkSquare–Release3.20currentverification-currenttotalinverification(itemsQAneedstotestandclose)
ãJeffSutherland1993-2010 198
• LightBlueX–Release3.20dailyopen–newtasksopenedperday
Figure34:ComprehensiveBurndownChartshowingdailytaskinflow/outflowandcumulativeprojectchurn[15].
Thecumulativeclosed(rightscale)ismuchhigherthanthestartingnumberofabout150tasks(leftscale).ThereasonforthisisthattheSprintBacklogminorchangesareconstantlycomingintotheSprintBacklogforthefollowingreasons:
• QAisfindingbugs,oftengeneratingmultipletasksthatcanbeclosedwithonedeveloperfix.
• Productdevelopmentisaddingtasksprimarilybecauseofcustomersmovinginandoutofthemixforgo-liveatendofSprint(thisisnotallowedinTypeAandBSprints).
• Developmentisdiscoveringnewtasksastheyfleshouttechnicaldesign.
ThecumulativeclosedtasksisanindicatorofthechurnonaprojectandthereasonwhyBrooks[5]notesthatdevelopmentalwaystakethreetimesaslongasinitialestimates.Automatedreportingandrapidturnaroundcanradicallyreducetimetocompletenewtasks.NotethestrongdownwardvelocityontheBurndownChartdespiteprojectchurn.PatientKeeperwasabletomovequicklyintothemarketplaceandachieveleadershipinthe healthcare mobile/wireless market [11] through delivering over 45 production releases of thePatientKeeperPlatformin2005forlargeenterprisessuchasPartnersHealthcareinBoston,JohnsHopkinsin
© Jeff Sutherland 1993-2007
Comprehensive Burndown Chart
320 PR Burndown
020406080
100120140160180
5/9/20
05
5/16/2
005
5/23/2
005
5/30/2
0056/6
/2005
6/13/2
005
date
PR c
ount
050100150200250300350400450
tota
l clo
sed
320 current open 320 current verification320 daily 'closed' 320 daily open320 total 'closed'
ãJeffSutherland1993-2010 199
Baltimore,andDukeUniversityHealthSysteminDurham.GartnerGroupputPatientKeeperastheleaderintheir“magicquadrant”fortheindustrysegment.TypeCScrumwasakeycontributortothissuccess.
FIGURE35:GARTNERGROUP“MAGICQUADRANT”FORHEALTHCAREMOBILEAPPLICATIONS[107].
5.CONCLUSIONSMovingtoaTypeCScrumisnotforthefaintofheart.ItrequiresScrumteamsthatcanexecuteastandardsprintflawlessly,anautomateddatacollectionandreportingsystemthatiseasytoimplementandupdate,andacorporateculturethatembraceschange.GoingtoaTypeCScrumwilltransformacompanyinanorganizationwhereScrumbecomesmissioncriticalfortheentireorganization,notjustsoftwaredevelopment.
GregorRothfussprovidedanexcellentsummarywhenseeingthereportingmechanismsforaTypeCScrumforthefirsttime[89]:
iwasaguestattheAgileroundtablenearBostonlastnight.Theeventdrewacrowdofveteransoftwareengineers,iwastheyoungestinattendancebyabout20years.
kenschwaberoutlinedhisandjeffsutherland'sScrumapproach,whichstruckmeasinterestingandworthwhiletofollowupon.
Leaders Challengers
Niche Players Visionaries
PatientKeeper
Allscripts Healthcare Solutions
Epic Systems
McKesson
MDanywhere Technologies
MedAptus
ePhysician
MercuryMD MDeverywhere
ePocrates
Medical Information Technology
(MEDITECH) Siemens
Eclipsys Technologies
QuadraMed Ability to
Execute
Completeness of Vision
ãJeffSutherland1993-2010 200
jeffsutherland,CTOofpatientkeeper,demonstratedhowhemanageshisteamsofdeveloperswithGNATS. jefffigured thatdevelopers loathe red tape,andhad thegoal to limit theeffort required to1minuteperday fordevelopers,and10minutesperdayforprojectmanagers.
and he was not using gantt charts to achieve this either. calling gantt charts totally useless for projectmanagementbeyondgivingwarmfuzziestotheclient,heexplainedhowheleveragedtheirbugtrackertodoubleasameanstokeeptrackofeffort.
eachmorning,developersreviewtheirtasksandupdatetheworkremainingestimateswhichhaveagranularityofoneday.theprojectmanagers,inturn,analyzethereportsthatGNATSautomaticallycreates.reportssuchasnumberofnewtasksvs.closedtasks,totalworkremainingandothermetricsthatcanbederivedfromthetaskdata.
tasksarethecornerstonehere.jeffwasabletodemonstratetothebusinesssidethatthehighlevelbusinessgoalswereoffby100%withtheireffortestimates,whilethelow-leveltasksachievedanaccuracyof10%onaverage.thisledtoenthusiasmfromallpartiestodrilldownonanyprojectandgettothetasklevelASAPtogetmeaningfulestimates.and,likepsychohistory,projectmanagementisinherentlystochastic.
‘nowheretorun,nowheretohide’
theleveloftransparencyofthissystemisunprecedented.witheveryoneinthecompanyabletoseeonadailybasishowmuchworkwasremainingandwhattheroadblockswere,theinitialfearsthatdeveloperswouldbepoundedon bymanagement turned out to be unfounded. instead, the transparency enables everyone to do real-timeadjustmentsandtodetectproblemsearly,whichhastakenalotofpoliticsandsecond-guessingoutoftheequation.
whenanalyzingaproject,jefffocusesonburndown,thepartofareleasewhereopentasksarerelentlesslydrivendownto0byajointeffortofdevelopersandbusinesspeople.thecorrespondinggraphic(roughlyabellcurve)illustratestheimportanceoftheburndownnicely,addingweighttojeff'sassertionthatburndownistheonlythingthatmatterstogetareleasedoneintime.
“whichpromptedmetoaskforadviceonhowtodriveanopensourcereleaseasareleasemanager.peoplearenotexactlyrequiredtodoyourbidding,butmetricsmayhelptheretoo.collecttheseusefuldatapoints,asthebugzilla-bitkeeper integration is doing, and let them speak for themselves. peer pressure and pride inworkmanshipwilltakeoverfromthere.that'stheideaanyway…”
KeyfeaturesmentionedintheRothfussreportare:
• Unprecedentedtransparency
• Companywidevisibility
• Metricsdrivendecisionmaking
• Peerpressureandprideinworkmanshipdrivingproductivity
TypeCScrumincreasesspeedofdevelopment,alignsindividualandcorporateobjectives,createsaculturedrivenbyperformance,supportsshareholdervaluecreation,achievesstableandconsistentcommunicationofperformanceatalllevels,andenhancesindividualdevelopmentandqualityoflife.
ãJeffSutherland1993-2010 201
CHAPTER7:CASESTUDIESOneofthemostinterestingthingsaboutScrumistheuniquecasestudiesthathavebeenpublishedatIEEEconferences. Scrum is used by some of themost productive, highmaturity, andmost profitable softwaredevelopmentteamsintheworld.Itpowers:
• Themostproductivelargedevelopmentproject(overamillionlinesofcode)everdocumented.
• ThemostuniqueCMMILevel5implementationontheplanet.
• Themostprofitablesoftwaredevelopmentprojectinthehistoryofsoftwaredevelopment.
TheSirsiDynixprojectreimplementedalargesystemthatsupportedover12,500librariesin42countrieswith120millionusers.Inthemiddleoftheproject,thedevelopmentteamwasdoubledinsizewithteamsfromStarSoftDevelopmentLabs inSt.Petersburg,Russia.TheirvelocitymorethandoubledthedaytheRussianteamscameonline.ThelinesofcodedeliveredperdeveloperwasashighasthebestsmallteamcollocatedScrumprojects.
SystematicSoftwareEngineeringinAarhus,Denmark,spentsevenyearsandover100,000personhoursofprocessengineerstoachieveCMMILevel5certification,reducereworkby80%,andimproveproductivityby31%.WithinsixmonthsafteraScrumCertificationcoursetheyhadreducedplanningtimeby80%,defectsby40%,totalcostofaprojectby50%whilesimultaneouslyenhancingcustomerandemployeesatisfaction.TheynowbidScrumprojectsat50%ofthecostofwaterfallprojects.
OneofthemostinterestingScrumimplementationsisGoogle’sAdWordsimplementations.ThisapplicationdrivesthemajorityofGooglerevenuegrowthandhelpscreatemarketcapitalizationthatishigherthanIntelandjustbelowthatofChevron,themostprofitableoilcompanyintheworld.TheAdWordsproject,poweredbyScrum,hasdistributedteamsinfive locationsandinterfaceswithvirtuallyallGoogleproductsoneveryrelease.Asaresult,theGoogleprojectmanagerneededtoinsertmorestructurethanisusuallyassociatedwithGoogle teams. His seamless introduction of Scrum based on resolving the highest priority impedimentsobservedbytheteamsresultedinanimplementationthatnolongerneededaScrumMastertofunction.Theteamsranbythemselves.
ãJeffSutherland1993-2010 202
SSH!WEAREADDINGAPROCESS…(ATGOOGLE)MarkStriebeck,GoogleInc.
Abstract
Googleisverysuccessfulinmaintainingitsstartupculturewhichisveryopenandengineering-centric.Projectteamsdon’thaveaprojectmanager,butorganizethemselvesandcommunicatedirectlywithallstakeholders.Most featuredecisionsaremadeby theengineering teams themselves.Aswellas thisworks forproducts likesearch, gmail… it creates issues for theAdWords frontend (AWFE)application.AWFE ismuchmoreproductmanagementandreleasedatedriventhenotherGoogleapplications.ThispresentationdiscusseshowwecarefullyintroducedagilepracticestocoordinatetheAWFEdevelopmentteamsandmadetheprocessmoreefficientandpredictable.
1.INTRODUCTIONGoogleisknownforitsstartupcultureanditseffortstomaintainit.IntermsoftheAgileManifestoGoogleisalmostentirelyonthe“lefthandside”(withtheexceptionof“WorkingSoftware”).Traditionally,projectteamsdonothaveaprojectmanager,butorganizethemselvesandcommunicatedirectlywithallstakeholders.Evennow,whereGooglehasmorethen6000employeesinnumerousofficesaroundtheworld,Googleisstillveryengineeringdriven.Manynewproductideascomefromthe20%projects5ofitsemployees.
TheoverallmindsetatGoogleistohaveaslittleaspossiblestandardprocessesaspossible.Thereasonisthattheindividualengineeringteamswillknowbestwhatisrightforthem.Uppermanagementontheothersidehastrustinitsengineersthattheywouldnotabusethisautonomybutdowhatisbestfortheirprojectandthecompany.
AdWordsisdifferent.BeingaB2Bapplication,meansthatitneedsmuchmorebusinessfocus,salesmaterialhas tobeupdated, supporthas tobe trained, and external communication aboutmajor featureshas tobeprepared(forums,blogsandemails).
ThereforeAdWordshadafewstandards:
• Fromtheinitialproductidea,theproductmanagertogetherwithaUIdesignerandusabilityspecialistscreates almost final UI mockups. These mockups are used for a final project review by seniormanagementandthengiventoengineeringforimplementation.
• Duringthewholeprojectlifecycle,theproductmanagerholdsweeklymeetingswithallstakeholders(engineering,QA,UI, support,marketing).These core teammeetings are themain communication
5EveryGoogleemployeeisencouragedtospend20%ofhis/hertimeonapersonalproject.Thisprojectshouldnotbetoocloselyrelatedtotheemployees’actualwork.
ãJeffSutherland1993-2021 203
channel.Allmajorproductdecisionsaremadeoratleastdiscussedhere.Lotsofchangerequestscomefromthesecoreteammeetingsduringtheprojectlifetime.
• Although,thecoreteamsetsinitialreleasedates(withinputfromengineering),thefinalreleasedateisdeterminedbyengineeringprogressandqualityofthecode.GiventhescaleofAdWords(numberofusers, business relevance, load, infrastructure); a small bug can have very severe consequences.Thereforefeaturesareratherdelayedthenreleasedwithinsufficientorevenunknownquality.
• This level of process worked well in the beginnings of AdWords. But the AdWords productdevelopmentoutgrewthisultralightweightprocess
• Theapplicationcodeconsistsofmorethen500KLOC
Theengineeringteamisdistributedin5officesworldwide–thereareconstantly15-20majorprojectsongoingplusmaintenanceandsmallimprovements.
AndtheAdWordsapplicationanddevelopmentteamisstillgrowing…
Theunpredictabilityoflaunchdatescausedmoreandmoreconcern.Nobodywantedtolowerthehighqualitystandards.Buttheinitialreleasedatesneededtobemorereliableanddelaysshouldatleastbeknownandcommunicatedmuchearlier.
BecauseofitssizeandcomplexityAdWordshasfairlylargemanagementteam(forGooglestandards).Inordertobeeffectivethemanagementteamneededmuchmorevisibilityintotheprojectsandtheirstatus.
Finally,therateofchangeisveryhighinAdWords.Teamswhoworkonaprojectforafewmonthsmightfindthattheyhavealotofcleanuptodobeforetheycanfinallylaunch.Notsomuchbecauseofcodeintegrationissues(theAdWordsteamrunsafairlycomprehensivecontinuousintegrationsuite)butbecauseoffeaturechanges.Often,projectsthatrunforalongtimehavetoplaycatch-upwithallthefeaturechangesbeforetheyrelease.Inafewcasesthisleadtosignificantdelays.
2.FIRSTAGILEATTEMPTSTryingtointroduceaprocessinastart-upenvironmentsuchasGoogleoftenmeetsresistance.Becauseofthegoogleywayofdevelopingsoftware,manyengineerssimplydonotbelievethatanyformalprocesscanhaveabenefitbutwillonlyslowthemdown.
WhenItookonmyfirstprojectsatGoogleIwasjustafewmonthswiththecompany.Theengineersdidnotknowmeatall.ButitwasinterestingtoseehowtheGoogleculturehelpedmehere:AbigpartofGooglecultureis trust. This goes through thewhole organization. And although I was new to Google and AdWords, theengineersandPMstrustedmethatIwoulddotherightthings.Orbetter:theytrustedthepeoplewhohiredmethatIamsomeonewhowoulddoagoodjob.
So,mystrategywastogetaslittleinvolvedaspossibleintheactualcodingpartandtostartwithafewpracticesthatwouldjusthelpustotrackprogressandshowissues.Thenwewouldintroduceindividualagilepracticestofixsuchissuesduringdevelopment.Idecidedtostartwiththefollowingpractices:
ãJeffSutherland1993-2021 204
Areleasebacklogandburndowncharts.Thesetwotoolsprovidehighvisibilityintothedevelopmentprogressfortheprojectteam,butalsooutsiders.Usingsimplewikipagestostorethebacklogsallowedtheengineerstoupdatetheirprogressinverylittletime.Idecidedtomeasuretheburndownratebytaskscomplete,notfeaturecomplete.Measuringprogressinfeaturecompletehasmanyadvantagesbutalsoforcesateamtochangetheirdevelopmentprocessalot.ItwasoneoftheareaswhereIdecidedtoratherintroducethispracticelaterinordernottooverwhelmtheteam.
In past projects I made very good experience with estimating features/tasks in points. Especially in anenvironment like AdWords, where engineers are often interrupted by meetings or tech talks, real timeestimatesareaproblem.Iftheburndowngraphtellsusthatweareimplementing3daysofworkperweekthenitoftenleadstodiscussionswhattheteamisdoingtheother2days.Orpeopletrytomatchtheirupdatestorealdays.Pointsareagoodabstractionlayerthatavoidsanysuchdiscussion.
Scopechangesareincludedinacontrolledwaybyfirstestimatingthem,addingthemtothebacklog.Here,theburndownchartshelpedtremendouslytogetaquickassessmentoftheimpact.
Aweeklydevelopmentcheckpointmeetingtoplanthenextweekandworkonscopechanges.Thesecheckpointmeetingswereattendedbytheengineers,QA,PMandUI.AtthispointIdidnotintroducerealiterations.Mypersonalexperiencewasthatchangingtoiteration-baseddevelopmentisasignificantchangefordevelopersandQA.Itsoundedtooheavytointroduceatthispoint.
Fortheadoptionofthesepractices,Itriedveryhardnottoimplementanythingtop-downbuttogetbuy-infromengineersandtheproductmanagers.Theinitialchangessoundedreasonabletotheengineers.BecauseIwasmanagingseveralprojects, Icouldnotbetooclosely involved inthedevelopmentactivities itself.Thisprobablyworkedtomyadvantage–theengineersrealizedquicklythatIwouldnottrytotellthemhowtodotheirjob,butthatIonlystructuretheprojectinacertainwaywhichwasnottoointrusive.Also,oneofthegoalswastokeeptheself-organizingcharacterofteamsintact.Afterall,thisisabigpartofGooglecultureandouragile adoption approachwould have failed ifwe had severely impacted it – nomatter how successful theprojectswouldhavebeen.
Thisapproachalsohelpedmetoworkwithseveralprojectsatthesametime.ManymeetingsregardingUI,features,design…tookplacewithoutme.Onlywhenwediscussedscope,schedulingorplannedthenextsteps,Iwasthereandwasusuallyleadingthemeeting.
2.1.THEGUINEAPIGPROJECTSChangesatGoogleareoftendoneinsomekindofguerillaapproach:oneprojectteamadoptssomethingnew.If itworks, other project teams get interested andwill try it aswell. Therefore,we started onlywith twoprojects:
ProjectA:Thiswasaverynewpieceoffunctionalitywhichdidnotoverlapwithexistingfeatures.TheUIwasfairlycomplex;theengineeringteamconsistedofnewrecentcollegegraduatesworkinginaremoteoffice.
ProjectB:ThisprojectwasasimplifiedversionofAdWords.Itwasheavilyintegratedintoexistingfeatures(webasicallyhad to think about everyother feature andhad to integrateordisable it). The teamconsistedofexperiencedengineers.SomeofwhichhadalreadyworkforsometimeatGoogle,otherswerenewtoGoogle).
ãJeffSutherland1993-2021 205
2.2.THEFIRSTPROCESSSTEPSIn both projects, we used the UI mockups to generate the release backlog by dissecting the screens intoindividual features. This pre-development process is very well established at Google and it seemed toocomplicatedtomakethispartmoreagile.
Thereleasebacklogswerestoredinwikipageswhichmadeitveryeasyforengineerstoupdatethem.Fromthesewikipagesweautomaticallygeneratedburndowngraphstovisualizetheprojectprogress.Theconceptofgivingstatusupdatesinworkleftandnotinworkcompletedwasinitiallystrangetobothteams.Buttheengineersquicklyrealizedtheadvantage.
Asstatedearlier Ididnot introduce iterationsat this time. Instead I installedweeklycheckpointswith thedevelopment team (PM, UI, Engineering and QA). In these checkpoint meetings, we discussed progress,additional feature requests and other issues. Additional featureswere estimated and added to the releasebacklog.Iextendedtheburndowngraphsandusedavariablefloortoindicatethescopechanges.Thegraphsgaveusquickfeedbackwhattheestimatedimpactoftheseadditionalfeatureswas.
TABLE1:BURNDOWNGRAPHWITHVARIABLEFLOOR
AlthoughIdidnottrytoimplementanimmediatetestingofimplementedfeatures,Iwantedtogetawayfromthepurelyphasedapproachwhere testing starts afterdevelopment is finished.Topush for this,we setupstagingserversthatwererebuiltonanightlybasewiththelatestcode.ThesestagingserverswereusedfortestingtheapplicationbutalsoforUIwalkthroughs6.
Usually,theyareperformedtowardstheendofaprojectwhenthesystemisnearlycomplete.Butbecausewestagedtheapplicationearlyonandimplementedend-userfeatures(fromtheUImockups)wecouldstartwiththeseUIwalkthroughsmuchearlierandgatherimportantfeedbackforthefurtherdevelopment.
6UIwalkthroughsarelivedemonstrationsofthesystemwiththewholecoreteamtogatherfeedbackanduncoverusabilityissuesearlyenough.
ãJeffSutherland1993-2021 206
2.3.ISSUESTOOVERCOMEInbothprojectswefacedsimilarissues:
Customer/ProductOwnerconcept
Mostagileprocesseshavethisrolewhichisresponsibleforfeatures,prioritizationandultimatelyscopevs.releasedatedecisions.Itisusuallyanindividualorteamoutsideofthedevelopmentteam.ButatGoogle,manyoftheseresponsibilitiesrestwiththeteamleads.Theproductmanagersusuallyhavemorethen10projectsatthesametime.Thisdoesnotgivethemthebandwidththattheproductownerrolerequires.Also,theytrustthe tech leadsandUIdesignersenough that theywillmakegooddecisions (often,when Iaskedaproductmanagerforprioritizationofafeature,heturnedtohistechleadandsimplyasked“whatdoyouwanttodo?”).
Thisgivestheplanningandprioritizationmeetingsadifferentdynamic.Often,thetechleadsdonotseetheneedtomakesuchdecisionsduringtheplanningmeetingsastheyknowthattheywillbe involvedenoughduringdevelopmentitselfthattheycanmakesuchdecisionsatalaterpoint.Iusuallydrovetheteamtomakeatleastthosedecisionswhicharenecessarytocreategoodeffortestimatesandprioritiesforthebacklog.Ialwayswantedtoleavetheweeklycheckpointmeetingswithgoodupdatestothereleasebacklog.
Retrospectives
Forme,frequentretrospectivesbecamesuchanimportantpartofsoftwaredevelopmentthatItriedtoinstallthemintheweeklycheckpointsfromthebeginning.Itwouldhavehelpedalotwithimprovingourprocessthroughconstantfeedback.
Butbothteamswerenot(yet)usedtohavingaformaldevelopmentprocess.Theweeklyretrospectivesusuallyturnedintoastatusreportfromthelastweekbutverylittleabouttheprocessitself.ThiswasaggravatedbytheengineeringcentriccultureatGoogle.Whenanissuecomesup,mostengineersatGoogleonlyconsidertechnologytofixit.
Afterafewweeks,Isilentlydroppedretrospectivesfromtheweeklycheckpoints.Idecidedtowaituntiltheteamsembracedtheconceptofadevelopmentprocessandthattheyownitandcouldchangeittofixproblems.
Constantscopeincrease
Inbothprojects,thescopeincreasedsignificantly(morethen30%)duringdevelopment.Interestingly,thesescopechangeswerenot theresultofadditional featurerequestsbytheproductmanagers.Mostadditionaltasksweretheresultsofoversightsduringthereleaseplanning:
TheengineeringteammissedfeaturesintheUImockupswhenwecreatedthereleasebacklog
IntegrationsintootherAdWordsfeatureswereoverlooked.Also,therateofchangeinAdWordsisveryhigh.Duringdevelopmentothersareasoftheapplicationchangedandwehadtochangeourintegrationaswell.
Mostof theseadditional taskscouldnotbedownprioritized fora laterreleasebuthad tobeaddedto therelease.
ãJeffSutherland1993-2021 207
Inbothprojects,thisleadtoseveralpostponementsofthereleasedateasnootherfeaturecouldbedroppedfromthefirstrelease.
Although, there was considerable frustration about these delays, both project teams and managementappreciated thatwe at least knew about these postponements early enough andnot just theweek beforerelease.Theburndowngraphsgaveagoodvisualizationandthereleasebacklogsmadeiteasyforeveryonetounderstandwhatwaslefttobeimplemented.
Thebacklogwashandyasthingscameupovertimeandaswediveddeeper.Onefunctionwastonotloosethelineitemsbutmoreimportantitwasusefulfortheteamtoseehowmanyunanticipatedissuescroppedupandhaveagoodsnapshotintime.
ProductManager
2.4.WORKINGWITHTHEREMOTETEAMAs stated earlier, project Awas implemented in a remote location. The rest of the core teamwas in ourheadquarters.Initially,Iwasconcernedhowthatteamwouldreacttomyleadership–iftheywouldappreciateitasmuchastheotherteamorifthewouldregarditasaheavy-handedapproachfromheadquarters.
TomysurpriseIdidnotencountermanyissueswiththisproject.Onlyprovidingtoolstogetmorevisibilityintodevelopmentprogressandfacilitatingplanningmeetingsseemedtobetherightleveltogivetheremoteteamenoughroomtoworkmostlyautonomously.Also,Icouldmakemyselfveryusefulinfacilitatinglotsofcommunicationwithotherengineersinourheadquarters.TheteamrealizedquicklythatIindeedtriedtohelptheprojectprogressandnottocontrolthemremotely.
3.ADDINGAGILITY–ONEPRACTICEATATIME3.1. DAILY STANDUP MEETINGS Bothproject teams initiallyrejectedthe ideaofdailystandupmeetings.Theywereseenasanunnecessaryoverhead.
Butduringdevelopmentwediscoveredissuesintheweeklycheckpointsfromthepastweeks:
QAtestedunfinishedfeaturesorwasnotsurehowtotestnewfeatures
Engineerswhoworkedonrelatedfeaturesworkedonthesamerefactors.TheAdWordsengineeringteamhasaveryhealthycultureofconstantlyrefactoringthecode.Thedownsideisthattwoengineerswhoworkonrelatedfeaturesoftenstarttoimprovethesamecode.
Engineers could not continue with their implementation because they depended on a task from anotherengineer.Oftenenough,theotherengineerwasnotawareofthisdependency.
Itwascleartoeverybodythattheseissuescouldhavebeenavoidedhadtheteamcommunicatedearlier.Atthispoint itbecameeasy to convinceboth teams to tryoutdaily standupmeetingsand to includeQA in thesemeetings.
ãJeffSutherland1993-2021 208
Thefirststandupmeetingswerequitelengthy.Everybodyhadalottotalkaboutandhadproblemstofocusjustonaquickstatusupdate(“done”,“to-do”,and“issues”).Butafterafewdaysnobodyhadabigbaggageanymoreandeverybodyrealizedthatthereisnotmuchtotalkifyourestrictyourselftothepast12hoursandnext12hours.Several issueswereresolvedorat leastuncoveredduring thesemeetings.Afteracoupleofweeks, both projects did not need a reminder anymore butmade the standupmeeting part of their dailyroutine.
3.2. SMALL STEPS – COMPLETELY FINISHING A FEATURE/TASK InprojectA,theprogresslookedverygood.Initially,weestimated3weeksforasetofscreens.Whenwedidlow-levelestimates,wecameto40points.Afterthefirstweek,theteamdid8points–inthesecondweek7.5points.Ilookedasiftheinitialestimatewastoolowandtheteamwouldneed5insteadof3weeks.
Interestingly,thetechleadoftheteamwasconvincedthatthescreenscouldstillbeimplementedin3weeks(i.e.allremaining24.5pointsin1week!)quote:“Itjustdoesnotfeelthatmuchanymore”.
Afterweek3,theteamwasnotdone.Theteamimplementedanother9points.Thevelocitylookedverystable:~8pointsperweek.
Tomybigsurprise,thetechleadannouncedinthecoreteammeetingonceagainthathisteamwillbedoneinoneweek…
Ittookmesometimetolearntotrusttheburndowngraphandtoquestionmygutfeelingwhenafeaturewouldbefinished.
TECHLEAD
Thefourthandfifthweekshowedasignificantdropinvelocity:4pointsand2.5points!Itturnedoutthattheteamdidnotcompletelyfinishthetasks:testswerenotwritten,codewasnotreviewed(whichismandatoryatGoogle),featureswerenotcompletelyintegrated.Thiscausedtheburndowngraphtogodownbecausewedidnotmeasureprogressinfinishedfeatures,butintasks.
This causeda furtherdelayand thescreenswere finally implementedafter7weeks.Thisadditionaldelaycausedsomeconcernwiththecoreteam.ToavoidthissituationIaddedagreen/yellow/redcolorcodingtotheburndownchartstoindicatehowmanytasksarenew/started/finished.Thismadeitveryclearifvelocitywas high becausemany features are partially finished or if the team completely finished a feature beforemovingtothenextone.
ãJeffSutherland1993-2021 209
FIGURE2:INDICATINGSTARTEDANDFINISHEDTASKS
Theteamrespondedverypositively.Itwasquiteashockfortheengineerstoseethatupto80%ofalltaskswereina‘started’state.Theystartedtokeepthecorridorofstartedtasksassmallaspossible.
Overall,thiswasaveryhealthylearningexperiencefortheteam.Itshowedthemthedifficultythatwetendtohavewhen trying toestimatea releasedate insteadofderiving the releasedate fromeffort estimatesandprogress.Italsoshowedthemthatwecanonlymeasureprogresswell,ifwecompletelyfinishtasksandnotleavesmallbitsandpiecesaroundwhichsometimesturnouttobemuchlargerthenwethought.
3.3. SPIKES Intheweeklycheckpointmeetingsweoftendiscoveredthattaskstookmuchlongertheninitiallyestimated.Ortheteamhadproblemswithestimatinganewfeature.
Initially, the engineers justwanted to pad estimates for such unknown tasks. Often enough, these paddedestimatesweremuchtoohighorstilltoolow.Andeverybodycouldseethattheyloweredtheusabilityofourburndowngraphssignificantly.So,weaddedinaspike(aninvestigativetask)tohelpdeterminewhattheeffortfortheimplementationwouldbe.Especiallywhenthescopecontinuedtogrow,everybodyrealizedthevalueofgettingabetterestimateofimplementingafeaturebeforeactuallystartingtoworkonit.
4.RELEASEEXPERIENCEThetwoprojectshadsomewhatdifferentreleases:
ProjectA)
The team had fixed many bugs already during development, only few bugs were discovered in the finalintegrationtestphase.Itwasaverysmoothlaunch.
ProjectB)
BecauseoftheintegrationintoallotherAdWordsfeatures,QAfoundmanyissuesduringdevelopment–mostofthemthroughexploratorytesting(i.e.notreallytiedtoaparticularproductfeature).Theteamtriedtokeep
ãJeffSutherland1993-2021 210
thebugbacklogundercontrolbutdidnotwanttofixallbugs.Whenwecameclosetolaunch,wehadtoreviewthebugbacklogseveraltimesanddownprioritizemanybugs.Untilafewdaysbeforelaunchitwasnotclearifwecouldfixenoughbugstoreleaseit.
Atleasttheteamdidnotencounteranyissuesthatrequiredacompleteredesignofsomearea–whichcouldhaveeasilyhappenedforsuchafarreachingfeature.
Still,theoverallreleaseexperiencewasverypositive.Bothprojectswereverysuccessfulinproductionandhadveryfewissues.
5.FEEDBACKANDNEXTSTEPSIheldpost-mortemmeetingswithbothprojects.InthesemeetingsIfocusedtheteamsonlistingpositivesandnegativesandnotjumpingtodiscusssolutionsimmediately.Fromtheoveralllist,theteamsselectedtheworstissuesandbestpracticestokeep:
Positive
ProjectManagementandtools(burndownchartsandbacklogs)
EarlyQAandavailabilityofastagingserver
Teamworkandcollaboration
Negative
Unclearornonexistentprioritization
Feltasifteammissedreleasedateseveraltimes
Tooriskyatendbecauseofbugbacklog(ProjectB)
Itwasveryencouragingthatbothteamsfoundtheoverheadofmaintainingandupdatingthereleasebacklogsworthdoing.
Burndownchartsmadeiteasytoseewhenweremakingprogress,andgaveusanicesenseofsatisfactionandcompleteness.
ENGINEER
And,furthermorethattheprocessdidnotimpactthegreatteamworkandcollaborationthatGoogleteamshave.Also,theeffortofmaintainingadedicatedstagingserverwasappreciated.TheengineersfrombothteamswereverypositiveabouttheearlytestingandfeedbackbyQAthatthestagingserverafforded.
ãJeffSutherland1993-2021 211
Ithinkittooksometimegettingusedtotheapproachoftestingsoearlyindevelopment,andalsomakingsurethatQAanddevwereonthesamepage.IthinkthatourdailystandupsandalsohavingQAco-locatedwithdevhashelpedgreatlyhere.
ENGINEER
6.THESECONDVERSIONFromthefeedbackofthepost-mortemmeetingItriedtomodifythedevelopmentprocessfurthertoaddresstheworstissues.
InbothteamsIgaveatthispointapresentationaboutafullScrumprocess.DuringthefirstprojectsthereweremanytechtalksatGoogleaboutagiledevelopment(byinternalandexternalspeakers).Bothteamsgotveryinterested in it. They could see that their practices fit into agile development but heard a lot about otherpracticestoo.Also,theverypositivefeedbackofmyprojectmanagementstyleandtoolsshowedmethattheengineerstrustedmeandmyguidance.Inbothteamswediscussedwhichadditionalpracticestoadopt:
Product/ReleaseBacklog
Toaddress theprioritization issue, Iworkedwith theproductmanagersofbothprojects toorganize theirrequirementsinprioritizedlists.Ittookalittlebitoftimeforthemtogetusedtoit,butwasnotamajoreffort.Thecoreteammemberslikedthebacklogsalot.Itgavethemmuchmorevisibilityandinputintodevelopment.Initially,therewasstillthedesiretomakeeachfeaturehighpriority.Butsooneverybodyrealizedthatevenifafeatureisnotincludedinthecurrentiteration,itwillstillgetdonefairlysoon.
Iterationbaseddevelopment
Thiswasthehardestpracticetointroduce.Withoutpracticalexperienceitishardtoexplainwhyiterationsarebetterthanschedulingthewholereleaseatonceandaddingtoitwhennecessary.
Butwith the feedbackaboutmissingdeadlinesand toomanybugs, I couldexplainhowan iterationbasedapproachwouldaddressthese.Theconceptofnotonlyimplementingbutalsotestingandcompletelyfixingfeatureswithinthesameiterationsoundedveryappealingtotheengineers.Although,theyweresomewhatskepticalofthishigh-qualityapproach,bothteamswantedtogiveitatry.
Theteamssoonrealizedtheadvantages.Theplanningmeetingsbecamemuchmorefocusedthantheweeklycheckpointmeetingsfromthepreviousprojects.Notimewaswastedwithdiscussingthesamefeaturefor5weeksbutneverimplementingit.Ortodiscussanddesignafeaturethatfinallygetsdropped.
Weagreedtostartwith2weekiterations.Thissynchronizeswellwiththe2weekreleasecycleofAdWords.Wearefinishingtheiterationswiththecodefreezeforthenextpush.Thismeansthatahigh-priorityfeaturethatgetsputontheproductbacklogcanbeimplementedandreleasewithin4weekswithoutanyinterruption.
ãJeffSutherland1993-2021 212
FIGURE3:SYNCHRONIZEDDEVELOPMENTITERATIONSANDRELEASECYCLES
Retrospectives
Afterthepreviousprojects,bothteamshadsomeexperiencewithadefineddevelopmentprocessandthattheycaninfluence/changeit.Istartedtheiterationplanningmeetingsagainwitharetrospectiveandthistimeitwasmuchmore fruitful.Most contributionswere about howwedevelop our application andhowwe canimprovethat.
REVIEWOFITERATIONFEATURESWITHCORETEAMIn the first projects,we reviewed the applicationby clicking through it during the core teammeeting andcollectedsomefeedback.Now,withtheiterationbaseddevelopmentwedothesereviewsattheendofeachiterationandonlyonnewlyimplementedfeatures.Thismadethereviewsmorefocusedandgivesusfeedbackearlyenoughsothatwecanintegrateitinthenextiteration.
TESTINGTASKSFORFEATURESINSAMEITERATIONInordertotestfeaturesinthesameiterationastheyaredevelopedin,weaddedtestingtaskstotheiterationbacklog.TheQAengineerswereaskedtoprovideeffortestimatesforthesetaskssothattheycanbeincludedintheburndownchart.
Overall,theteamscouldseehowtheseprocesschangeswouldaddressthenegativefeedbackfromthepost-mortemmeetings.Bothteamsdidnotfullyunderstandhowthesepracticeswouldworktogetherbutagreedtogiveitatry.
AtthispointItookonathirdprojectwhereIimplementedthenewprocessfromthebeginning.TheproductmanagerofthisteamwasfromProjectA,theQAengineerfromProjectB.Thismadetheadoptionmucheasier.Also,many people in AdWords had heard about how I ranmy projects and the barrier to try it out wasconsiderablylower.
6.1. THE WORLD IS BETTER, BUT … Overall,themoreagileprocessesworkedreallywell.Everybodynoticedthattheadditionalstructurecomeswithverylittleoverheadandfixesmanyoftheissuesthatwehadbefore.
We'restillgettinguptospeedontheiteration-baseddevelopment.It'sbeennicefordevelopment,nowthatouriterationsareinsyncw.AdWordscodefreezecycle.ItwashardatfirstforUI/PM,buthasgotteneasierasPMhasassembledfartherprojectingroadmap,togiveUIacluewhatwillbeneededforacomingiteration.
TECHLEAD
Afteramonthortwo,bothproductmanagersrealizedthattheyneedtoestablisharequirementprocessthatensuresthatwenotonlyimplementlittlebitsandpiecesatatimebutkeeptheoverallrelease.Thisisanissue
Push 2CodeFreeze –Push 2
TranslationDeadline –Push 2
Push 1CodeFreeze –Push 1
TranslationDeadline –Push 1
week 1 week 2 week 3 week 4 week 5 week 6Pre-Iteration 1 Iteration 1
Pre-Iteration 2 Iteration 2Developmentiterations
Releasecycles
ãJeffSutherland1993-2021 213
that I hadwithprevious agile teams. I couldpersuade theproductmanagers todissect their releases intosmallerchunksandprioritizethem.
For these I created releaseburndowncharts to trackwhen theywill be finished.At thispoint I started tomeasureprogressonthereleaselevelinfeaturescomplete.Atthispointitwasveryeasytoconvincetheteamsthatthisistherightmeasurementasitwouldgiveusamuchbetterguidancewherethereleaseis.
Theteamsfirstthoughtthatitwasstrangetohaveoneiterationburndownchartandonereleaseburndownchart. But after a few iterations they saw the benefit of both. The iteration burndown to guide actualdevelopmentefforts.Andthereleaseburndowntoguidetheoverallreleaseplanning.
AnongoingissueistheQAinvolvement.IconstantlyhavetopushtheQAengineerstotestfeaturesimmediatelyafter they are implemented. The reason is that the QA engineers support several projects. And the otherprojectsarenotagile,i.e.don’trequiremuchattentionduringdevelopment,butalotattheend.ThismadeithardfortheQAengineerstoconstantlyspendalittlebitoftimeeachdayonorprojecttogivetheengineerstheimmediatefeedback.Rightnow,bothteamsquestionifitisworththeefforttoincludeQAtasksandeffortestimatesinourplanningasitdoesnotseemtohaveanybenefit.
Forme, it seems likeanextra taskofupdatinga tablewithdata (QAestimates) that’snotof significance forme.ButI’dreallyliketoknowifit’shelpfultoothers.Sofar,mostoftheestimateshavebeen0.1points.
QAENGINEER
Finally,theteamsdonottrytocreateareleasableproductattheendoftheiteration(whichisevenharderbecauseoftheQAissuementionedabove).Therearealwaystaskshalfimplemented,nottested,needreview…Fornow,Iamnotpushingtoohardonthis.TheteamscompletelyimplementenoughfeaturesperiterationthatwecanreleasethosewiththenextAdWordsupdate.
6.2. THE PROJECT MANAGER IS DISPENSABLE Recently,Iwentona3weekvacation.Iwasconcernedhowtheteamswouldcontinuewiththeagileprocessduringmyabsenceandremindersandreinforcementsofouragilepractices.
Butitturnsoutthattheteamsembracedtheprocessenoughtocontinueitevenwithoutanyreinforcement.Iterationplanningmeetingshappened,backlogswerecreatedaccordingtopreviousvelocity,anddailystandupmeetingstookplace…
7.WHEREAREWEGOINGFROMHEREWith the success of three project teamswe are now prepared tomakemuch bolder steps. Everybody inAdWordshadatleastheardabouttheadvantagesoftheagileapproach.Resistanceatthispointwillbemuchless.
• EstablishbacklogsandburndownchartsasstatusreportingstandardsinAdWords.Evenifteamsdonotadoptotheragilepractices,thesepracticesareeasytoimplementandprovideaverygoodvisibilityfortheteamsthemselvesbutalsomanagementandotheroutsiders.
ãJeffSutherland1993-2021 214
• Othermanagersvoicedinterest.WithashadowingapproachIwillguidethemthroughtheagileprocessandtrytogivethemenoughexperiencetoimplementagilepracticesintheirprojectsbythemselves
• AfewprojectsinvolveteamsfromseveralAdWordsdepartments(frontend,backend,NetAPI…).Suchprojectsalwaysrequiredmuchmoremanagementattention.AsgreatasGoogleengineersandtechleadsare,coordinatingandsynchronizingateamseffortswithotherteamsoftendistractstechleadstoomuch.Wewilleithertrytocoordinatetheseteamsasonebigteam(onebacklog,oneburndownchart)orusethe“Scrum-of-Scrums”approach.
• DuringthefirstmonthsatGoogleIheardfromotherdepartmentswhoareusingsomeagilepracticesorfull-on Scrum/XP processes. To support this effort we started a grouplet 7 that focuses on agiledevelopment.Wejustrecentlystartedthisgroupletandtheinitialresponse/interestwasoverwhelming–notonlyfromengineering,butalsoothergroups(QA,ProductManagement,UI,Usability)
• TheUIdevelopmentandusabilitypartofourdevelopmentprojectsisstillveryfrontloaded.Almostallofthisworkisdonebeforedevelopmentstarts.AfewusabilityexpertsandUIdesignersshowedinterestinmakingthisalsopartoftheiteration-baseddevelopment.
8.SUMMARYWiththehelpofanexperiencedagileleader(ScrumMaster,XPcoach…)itwaspossibletocarefullyintroduceagilepracticesintoGoogle-anenvironmentthatdoesnothaveanaffinitytoprocessesingeneral.Insteadofintroducingagrandnewprocess,individualpracticescouldbeintroducedeithertofixobservedissuesorjustto“trythemout”–thedevelopmentteamsrealizedtheadvantagesverysoon.
Along with these practices came a visibility into the development status that gave the approach greatmanagementsupport.
Allthiscouldbedonewithoutdestroyingthegreatbottom-upculturethatGooglepridesitselfof.Thepracticesonly affect how the projects are structured. Design and implementation remains fully an engineeringresponsibility.Withsomemodifications,wecouldevenkeeptheverystrongroleoftechleadsandUIdesigners.
In keeping the great culture and self-organization of the teams, I could easilymanage several projects inparallel.Icouldcontinuetorelyonallcoreteammemberstocommunicateeffectivelywithoutintroducinganyheavyprocesses.
7Googlegroupletsarecross-departmentgroupswhichfocusonaspecificareaofthesoftwaredevelopmentprocess(thereis a techdocumentation grouplet, a build tools grouplet…)Themembers of the grouplet use their 20% time for theirparticipation.
ãJeffSutherland1993-2021 215
5.REFERENCES
[1] H. Takeuchi and I. Nonaka, "The New New Product Development Game," Harvard Business Review, 1986.
[2] J. O. Coplien, "Borland Software Craftsmanship: A New Look at Process, Quality and Productivity," in 5th Annual Borland International Conference, Orlando, FL, 1994.
[3] A. C. Kay, "The early history of Smalltalk," presented at the The second ACM SIGPLAN conference on History of programming languages (HOPL-II), 1993.
[4] J. Collins, Good to Great: Why Some Companies Make the Leap... and Others Don't: Collins, 2001.
[5] R. A. Brooks, "Intelligence without representation," Artificial Intelligence, vol. 47, pp. 139-159, 1991.
[6] C. G. Langton, "Life at the Edge of Chaos," in Artificial Life II, SFI Studies in the Sciences of Complexity, Held Feb 1990 in Sante Fe, NM, 1992, pp. 41-91.
[7] C. Jones, Software assessments, benchmarks, and best practices. Boston, Mass.: Addison Wesley, 2000.
[8] M. Poppendieck and T. Poppendieck, Lean Software Development: An Implementation Guide: Addison-Wesley, 2006.
[9] M. Poppendieck, "PatientKeeper, the only company in the world that is not thrashing.," Boston, MA, 2007.
[10] J. Johnson, "Decision Latency Theory: It Is All About the Interval," Standish Group2018. [11] C. Alexander, The Timeless Way of Building, Volume 1, 1979. [12] H. Takeuchi and I. Nonaka, Hitotsubashi on Knowledge Management. Singapore: John
Wiley & Sons (Asia), 2004. [13] M. Poppendieck, "A History of Lean: From Manufacturing to Software Development," in
JAOO Conference, Aarhus, Denmark, 2005. [14] J. K. Liker, The Toyota way : 14 Management Principles from the World's Greatest
Manufacturer. New York: McGraw-Hill, 2004. [15] W. D. Holford and M. Ebrahimi, "Honda: Approach to Innovation in Aerospace and
Automotive/Pick-Up Truck Development: A Dialectical Yet Coherent Firm," in 40th Annual Hawaii International Conference on System Sciences (HICSS-40), Big Island, Hawaii, 2007.
[16] P. M. Senge, The Fifth Discipline: the Art and Practice of the Learning Organization. New York: Currency, 1990.
[17] K. Schwaber, Agile project management with Scrum. Redmond, Wash.: Microsoft Press, 2004.
[18] K. Schwaber and M. Beedle, Agile software development with scrum: Prentice Hall, 2002. [19] J. Sutherland, A. Viktorov, J. Blount, and N. Puntikov, "Distributed Scrum: Agile Project
Management with Outsourced Development Teams," presented at the HICSS'40, Hawaii International Conference on Software Systems, Big Island, Hawaii, 2007.
[20] J. Sutherland, C. Jakobsen, and K. Johnson, "Scrum and CMMI Level 5: A Magic Potion for Code Warriors!," in Agile 2007, Washington, D.C., 2007.
ãJeffSutherland1993-2021 216
[21] M. Striebeck, "Ssh! We're Adding a Process..." presented at the Agile 2006, Minneapolis, 2006.
[22] J. Sutherland, "Agile Development: Lessons Learned from the First Scrum," Cutter Agile Project Management Advisory Service: Executive Update, vol. 5, pp. 1-4, 2004.
[23] K. Schwaber, "Scrum Development Process," in OOPSLA Business Object Design and Implementation Workshop, J. Sutherland, D. Patel, C. Casanave, J. Miller, and G. Hollowell, Eds., ed London: Springer, 1997.
[24] K. Beck, Extreme Programming Explained: Embrace Change. Boston: Addison-Wesley, 1999.
[25] K. Beck, "Request for SCRUM Information,"15 May Boston: Compuserve, 1995. [26] M. Beedle, M. Devos, Y. Sharon, K. Schwaber, and J. Sutherland, "Scrum: A Pattern
Language for Hyperproductive Software Development," in Pattern Languages of Program Design. vol. 4, N. Harrison, Ed., ed Boston: Addison-Wesley, 1999, pp. 637-651.
[27] G. Benefield, "Rolling Out Agile at a Large Enterprise," in HICSS'41, Hawaii International Conference on Software Systems, Big Island, Hawaii, 2008.
[28] K. Schwaber, The Enterprise and Scrum. Redmond, WA: Microsoft Press, 2007. [29] J. Sutherland, A. Viktorov, and J. Blount, "Adaptive Engineering of Large Software
Projects with Distributed/Outsourced Teams," presented at the International Conference on Complex Systems, Boston, MA, USA, 2006.
[30] C. Jones, Applied software measurement : assuring productivity and quality, 2nd ed. New York: McGraw-Hill, 1997.
[31] M. Cohn, Agile Estimation and Planning: Addison-Wesley, 2005. [32] P. DeGrace and L. H. Stahl, Wicked problems, righteous solutions : a catalogue of modern
software engineering paradigms. Englewood Cliffs, N.J.: Yourdon Press, 1990. [33] F. P. Brooks, The Mythical Man Month: Essays on Software Engineering: Addison-
Wesley, 1995. [34] W. A. Wood and W. L. Kleb, "Exploring XP for Scientific Research," IEEE Software, vol.
20, pp. 30-36, May/June 2003. [35] C. Larman, Agile & Iterative Development: A Manager's Guide. Boston: Addison-Wesley,
2004. [36] S. J. Gould, The structure of evolutionary theory. Cambridge, Mass.: Belknap Press of
Harvard University Press, 2002. [37] M. Fowler, "Is Design Dead?," Software Development, vol. 9, April 2001. [38] J. Sutherland, D. Patel, C. Casanave, J. Miller, and G. Hollowell, Business Object Design
and Implementation: Springer, 1997. [39] K. Schwaber. (1995, 6 December 2005). Scrum Development Process. Available:
http://jeffsutherland.com/oopsla/schwapub.pdf [40] J. Sutherland, "Agile Can Scale: Inventing and Reinventing Scrum in Five Companies,"
Cutter IT Journal, vol. 14, pp. 5-11, 2001. [41] J. Sutherland, "Future of Scrum: Parallel Pipelining of Sprints in Complex Projects,"
presented at the AGILE 2005 Conference, Denver, CO, 2005. [42] F. P. Brooks, "No silver bullet - essence and accidents of software engineering," Computer,
vol. 20, pp. 10-19, April 1987.
ãJeffSutherland1993-2021 217
[43] G. Booch, Object-Oriented Analysis and Design with Applications: Benjamin Cummings Publishing Company, 1994.
[44] B. A. Ogunnaike and W. H. Ray, Process Dynamics, Modeling, and Control: Oxford University Press, 1994.
[45] J. Sutherland. (1996). ScrumWeb Home Page: A Guide to the Scrum Development Process. Available: http://jeffsutherland.com/scrum/original.html
[46] K. Schwaber, "Controlled Chaos: Living on the Edge," American Programmer, April 1996. [47] AberdeenGroup, "Upgrading to ISV Methodology for Enterprise Application
Development," Product Viewpoint, vol. 8, December 7 1995. [48] M. Pittman, "Lessons Learned in Managing Object-Oriented Development," IEEE
Software, vol. 10, pp. 43-53, Jan/Feb 1993. [49] G. Booch, Object Solutions: Managing the Object-Oriented Project: Addison-Wesley,
1995. [50] I. Graham, Migrating to Object Technology: Addison-Wesley, 1994. [51] B. W. Boehm, "A Spiral Model of Software Development and Enhancement," in
International Workshop on Software Process and Software Environments, Coto de Caza, Trabuco Canyon, California, 1985.
[52] J. Rumbaugh, "What Is a Method?," Object Journal, vol. Oct, 1995. [53] W. S. Humphrey, Introduction to the Personal Software Process: Addison Wesley, 1996. [54] H. Ziv and D. Richardson, "The Uncertainty Principle in Software Engineering," in
submitted to Proceedings of the 19th International Conference on Software Engineering (ICSE'97), 1997.
[55] P. Wegner, "Why Interaction Is More Powerful Than Algorithms," Communications of the ACM, vol. 40, pp. 80-91, May 1997.
[56] B. Boehm, "Project Termination Doesn't Mean Project Failure," IEEE Computer, vol. 33, pp. 94-96, 2000.
[57] D. K. I. Sobek, A. C. Ward, and J. K. Liker, "Toyota's Principles of Set-Based Concurrent Engineering," Sloan Management Review, vol. 40, pp. 67-83, 1999.
[58] W. S. Humphrey, A Discipline for Software Engineering: Addison-Wesley, 1995. [59] Matisse Software, "The Emergence of the Object-SQL Database," Mountain View,
CA2003. [60] W. D. Hillis, The Connection Machine. Cambridge, MA: MIT Press, 1985. [61] S. Levy, Artificial Life : A Report from the Frontier Where Computers Meet Biology, 1st
ed. New York: Vintage, Reprint edition, 1993. [62] C. Jakobson, "The Magic Potion for Code Warriors! Maintaining CMMI Level 5
Certification With Scrum.," Aarhus, Denmark: Agile 2007 paper in preparation, 2006. [63] K. E. Nidiffer and D. Dolan, "Evolving Distributed Project Management," IEEE Software,
vol. 22, pp. 63-72, Sep/Oct 2005. [64] R. Zanoni and J. L. N. Audy, "Projected Management Model for Physically Distributed
Software Development Environment," in HICSS'36, Hawaii, 2003, p. 294. [65] J. Barthelemy, "The Hidden Costs of Outsourcing," MITSloan Management Review, vol.
42, pp. 60-69, Spring 2001.
ãJeffSutherland1993-2021 218
[66] B. Gorzig and A. Stephan, "Outsourcing and Firm-level Performance," German Institute of Economic ResearchOctober 2002.
[67] StandishGroup. (2003). 2003 Chaos Chronicles. Available: http://www.standishgroup.com/press/article.php?id=2
[68] J. Sutherland, "Future of Scrum: Parallel Pipelining of Sprints in Complex Projects with Details on Scrum Type C Tools and Techniques," PatientKeeper, Inc., Brighton, MAMay 30 2005.
[69] M. Cohn, User Stories Applied : For Agile Software Development: Addison-Wesley, 2004. [70] C. Jones, "Programming Languages Table, Release 8.2," Software Productivity Research,
Burlington, MA1996. [71] M. C. Paulk, C. V. Weber, B. Curtis, and M. B. Chrissis, The Capability Maturity Model
®: Guidelines for Improving the Software Process. Boston: Addison-Wesley, 1995. [72] M. B. Chrissis, Konrad, and Shrum, CMMI – guideline for process integration and product
improvement, 2002. [73] D. R. Goldenson and D. L. Gibson, "Demonstrating the impacts and benefits of CMMI,"
CrossTalk, October 2003. [74] D. D. o. Science, "Maturity of customer and suppliers in the public sector," ed, 2006. [75] M. Fowler and J. Highsmith, "The Agile Manifesto," Dr. Dobbs, July 13 2001. [76] M. C. Paulk, "Extreme Programming From a CMM Perspective," IEEE Software, vol. 34,
2001. [77] M. C. Paulk, "Agile Methodologies and Process Discipline," CrossTalk, 2002. [78] Surdu, "Army Simulation Program Balances Agile and Traditional Methods with Success,"
CrossTalk, 2006. [79] P. E. McMahon, "Lessons Learned Using Agile Methods on Large Defense Contracts,"
CrossTalk, 2006. [80] D. Kane and S. Ornburg, "Agile Development: Weed or Wildflower," CrossTalk, 2002. [81] Krasner and Houston, "Using the Cost of Quality Approach for Software," CrossTalk,
November 1998. [82] M. Diaz and J. King, "How CMM Impacts Quality, Productivity, Rework, and the Bottom
Line," CrossTalk, March 2002. [83] M. K. Kulpa and K. A. Johnson, Interpreting the CMMI: A Process Improvement
Approach. Boca Raton: Auerbach Publications, 2003. [84] J. Rose, I. Aaen, and P. Axel Nielsen, "The Industrial Age, the Knowledge Age, the Internet
Age: Improving Software Management," Aalborg University2004. [85] J. O. Coplien, "Personal issues caused over 50% of productivity losses in the ATT Bell
Labs Pasteur Project analysis of over 200 case studies," Lynby, Denmark, 2006. [86] T. Sulaiman, B. Barton, and T. Blackburn, "AgileEVM - Earned Value Management in
Scrum Projects," in Agile 2006, Minneapolis, 2006. [87] NetObjectives. (2005, 9 Feb). Managing the Work. Available:
http://netobjectives.com/resources/downloads/ManagingTheWork.pdf [88] P. M. Senge, The Fifth Discipline: The Art and Practice of the Learning Organization,
Revised Edition. New York: Doubleday, 2006.
ãJeffSutherland1993-2021 219
[89] G. Rothfuss. (2003, 6 December). Metrics for Agile Methods. Available: http://greg.abstrakt.ch/archives/2003/11/metrics_for_agile_methods.html
[90] R. Martens. (2005, 28 November). On Be(coming) Agile: SCRUM Gathering – Advanced SCRUM and Rally as an Agile Enterprise. Available: http://theagileblog.net/2005/11/scrum_gathering_advanced_scrum_1.html
[91] T. Poppendieck, "Roots of poor management practices," Boston, MA, 2006. [92] M. Fackler, "The "Toyota Way" is Translated for a New Generation of Foreign Managers,"
in New York Times, ed. New York, 2007. [93] D. K. Taft. (2005). Microsoft Lauds 'Scrum' Method for Software Projects. Available:
http://www.eweek.com/article2/0,1895,1885883,00.asp [94] K. Johnson, C. Jacobsen, and J. Sutherland, "Scrum and CMMI Level 5: A Magic Potion
for Code Warriors!," in SEPG, 2007. [95] E. M. Goldratt and J. Cox, The goal : a process of ongoing improvement, 2nd rev. ed. Great
Barrington, MA: North River Press, 1994. [96] M. Cohen. (2006, Sep 4) Make a Bet on General Motors. Forbes. 162. [97] A. MacCormack, C. Kemerer, M. Cusumano, and B. Crandall, "Exploring Tradeoffs
Between Productivity and Quality in the Selection of Software Development Practices.," IEEE Software, pp. 78-85, 2003.
[98] I. Jacobson, Object-Oriented Software Engineering: A Use Case Driven Approach: Addison-Wesley, 1992.
[99] J. Sutherland and W.-J. van den Heuvel, "Enterprise Application Integration and Complex Adaptive Systems: Could System Integration and Cooperation be improved with Agentified Enterprise components?," Communications of the ACM, vol. 45, pp. 59-64, October 2002.
[100] J. Sutherland and J. Coplien, The Scrum Book: The Spirit of the Game: Pragmatic Bookshelf, 2019.
[101] J. M. Osier, B. Kehoe, and Y. Svendsen, "Keeping Track: Managing Messages with GNATS, The GNU Problem Report Managment System Version 4.0," Free Software Foundation2002.
[102] J. M. Bland and D. G. Altman, "Statistics Notes: Regression towards the mean," BMJ, vol. 308, pp. 1499-, June 4, 1994 1994.
[103] W. E. McCarthy, "The REA Accounting Model: A Generalized Framework for Accounting Systems in a Shared Data Environment," The Accounting Review, vol. LVII, pp. 554-578, July 1982.
[104] G. Weinberg, Quality Software Management Vol. 4: Anticipating Change: Dorset House, 1997.
[105] J. H. Holland, Hidden order : how adaptation builds complexity. Reading, Mass.: Addison-Wesley, 1995.
[106] J. H. Holland, Emergence : from chaos to order. Reading, Mass.: Addison-Wesley, 1998. [107] K. Kleinberg and T. Berg, "Mobile Healthcare: Applications, Vendors and Adoption,"
Gartner Group26 November 2002. [108] S. Group, "There's Less Development Chaos Today," in Software Development Times, ed,
2007.
ãJeffSutherland1993-2021 220
[109] H. Kniberg, Scrum and XP from the Trenches. Stockholm: Crisp, 2007. [110] I. o. Nonaka and H. Takeuchi, The knowledge-creating company : how Japanese
companies create the dynamics of innovation. New York: Oxford University Press, 1995. [111] E. M. Goldratt, Theory of Constraints. Burlington, MA: North River Press, 1990. [112] W. Cunningham. (1999). Wiki Wiki Web. Available: http://c2.com/cgi/wiki [113] J. Sutherland. (1997). Jeff Sutherland's Scrum Log. [114] K. Schwaber. (1997). Ken Schwabers's Scrum Web Page.
ãJeffSutherland1993-2021 221
APPENDIXI:
SCRUM:APATTERNLANGUAGEFORHYPERPRODUCTIVESOFTWAREDEVELOPMENT
Mike Beedle, Martine Devos, Yonat Sharon, Ken Schwaber, and Jeff Sutherland
Can a repeatable and defined process really exist for software development? Some think this is not onlypossible but necessary, for example, those who favor the CMM (Capability Maturity Model) approach tosoftwaredevelopment[71].
However,manyofusdoingworkinthetrencheshavefoundovertimethattherepeatableordefinedprocessapproachmakesmanyincorrectassumptions,suchasthefollowing:
• Repeatable/definedproblem.Arepeatable/definedprocessassumesthatthereisasteporstepstocapturerequirements,butinmostcases,itisnotpossibletodefinetherequirementsofanapplicationlikethatbecausetheyareeithernotwelldefinedortheykeepchanging.
• Repeatable/definedsolution.Arepeatable/definedprocessassumethatanarchitecturecanbefullyspecified,butinrealityitisevolved,partlybecauseofmissingorchangingrequirements(asdescribedabove),andpartlybecauseofthecreativeprocessinvolvedindesigningnewsoftwarestructures.
• Repeatable/defined developers. The capabilities of a software developer vary widely, so that aprocessthatworksforonedevelopermaynotworkforanotherone.
• Repeatable/definedorganizationalEnvironment.Schedulepressure,priorities(e.g.qualityvs.pricevs.manpower),clientbehavior,andsoonareneverrepeatable,andbecausetheyarehighlysubjective,theyareveryhardtodefine.
Theproblemwiththeseassumptionsisthatthesevariablesdohavelargevariances.Inreallifeprojectstherearealwayslargedynamicvariationsthatcanhaveagreatdealofimpactontheoverallproject.Forexample,newlyfoundchangesintherequirementsduringanapplicationimplementation—atypicaloccurrence—mayaffect drastically a project’s schedule that assumed thatall the requirementswould be captured up front.However, removing this uncertainty is nearly impossible because of the nearly universal sources ofrequirementschange:businessrequirementsdrivenchanges,usabilitydrivenchanges,re-prioritizationdrivenchanges, testing driven changes, and so forth. This issue cannot be solved through improvedmethods for
ãJeffSutherland1993-2021 222
identifyingtheuserrequirements.Insteaditcallsforamorecomplexprocessofgeneratingfundamentallynewoperatingalternatives.
Inotherwords,onceweacceptthatthesedynamicvariabilitiesdoexist,weclearlyneedmoreadaptivewaystobuildsoftware.However,whatwefearisthatmostcurrentmethodsdonotallowustobuildsoftenoughsoftware: present methods and design paradigms seem to inhibit adaptability. Therefore the majority ofsoftwarepractitionersnowadaystendtobecomeexpertsatwhattheycanspecifyinadvance,workingwiththeunstatedbeliefthatthereexistsanoptimalsolutionthatcanbeplannedapriori.Oncetechnologyisadoptedbyanorganization,itotenbecomesaconstrainingstructurethatinpartshapestheactionspaceoftheuser.Thuswebuildsoftwaretoomuchlikewebuildhardware—asifitweredifficulttochange,asifithastobedifficult to change. Inmanyorganizations, “The systemrequires it” or the “Systemdoesnot allow it”havebecomeaccepted (andoftenunavoidable) justifications forhumanbehaviorbeforeandafter the system isreleasedtoproduction.
Incontrast,Scrumallowsustobuildsoftersoftware,sothereisnoneedtowritefullrequirementsupfronts.Sincetheusersdonotknowwhatispossible,theywillaskforthepre-tech-papersolutionthattheyperceivetobepossible(“lookingattherearviewmirror”).Butintruth,noteventhesoftwaredevelopersknowfullywhatcanbebuiltbeforehand.Therefore,theuserhasnoconceptofwhatispossiblebeforeheorshecanfeelitortouchit.Assuch,TheScrumpatternspresentedhereofferacollectionofempiricaltechniquesthatassumeup front the existence of uncertainty but that provide practical and specific techniques to tame it. Thesetechniques are rooted in complexity management, that is, in self-organization, management of empiricalprocesses,andknowledgecreation.
In that sense, Scrum is not only a “parallel iterative and incremental” development method, it is also an“adaptive”softwaredevelopmentmethod.
HOWDOESSCRUMWORK?
Scrum’sgoalistodeliverasmuchqualitysoftwareaspossiblewithinaseries(threetoeight)ofshorttimeboxes(fixed-timeintervals)calledSprintsthattypicallastaboutamonth.
Eachstageinthedevelopmentcycle(Requirements,Analysis,Design,Evolution,andDelivery)isnowmappedto a Sprint or series of Sprints. The traditional software development stages are retained primarily forconveniencetrackingmilestones.So,forexample,theRequirementsstagemayuseoneSprint,includingthe
ãJeffSutherland1993-2021 223
deliveryofaprototype.TheAnalysisandDesignstagemaytakeoneSprinteach,whiletheEvolutionstagemaytakeanywherefromthreetofiveSprints.
EditorsNote:Inrecentyears,releasecycleshaveshortenedtothreemonthsorlessformostsoftwareproducts.RequirementsarespecifiedjustenoughandjustintimetobereadyatthestartoftheSprintcycle.SprintsproduceworkingsoftwareforreviewattheendofeverySprint.Asaresult,Analysis,Design,andEvolutionoccurineverySprint.Sprintcyclesinmanycompanieshavebeenshortenedtotwoweeksorless.Inthebestcompanies,deliveryisincludedineverySprint[41].
Unlikearepeatableanddefinedprocessapproach,inScrumthereisnopredefinedprocesswithinaSprint.Instead,Scrummeetingsdrivethecompletionoftheallocatedactivities.
EachsprintoperatesonanumberofworkitemscalledaBacklog.Asarule,nomoreitemsareexternallyaddedintotheBacklogwithinaSprint.Internalitemsresultingformtheoriginalpre-allocatedBacklogcanbeaddedto it.Thegoalofasprint is tocompleteasmuchqualitysoftwareaspossible,buttypically lesssoftware isdeliveredinpractice.
Theendresultisthattherearenon-perfectreleasesdeliveredeverySprint.
DuringaSprint,ScrumMeetingsarehelddailytodeterminethefollowing:
• ItemscompletedsincethelastScrummeeting..
• Issuesorblocks thatneedtoberesolved. (TheScrumMaster isa teamleaderroleresponsible forresolvingtheblocks.)
• NewassignmentstheteamshouldcompletebeforethenextScrummeeting.
ScrumMeetingsallowthedevelopmentteamto“socializetheteammembers’knowledge”aswellasproduceadeepculturaltranscendence.This“knowledgesocialization”promotesaself-organizedteamstructurewithinwhichthedevelopmentprocessevolvesonadailybasis.
AttheendofeachSprintthereisademonstrationto:
ãJeffSutherland1993-2021 224
• Showthecustomerwhat’sgoingon.
• Givethedeveloperasenseofaccomplishment.
• Integrateandtestareasonableportionofthesoftwarebeingdeveloped.
• Ensurerealprogress,thatis,thereductionofBacklog,notjusttheproductionofmorepaper/hoursspent.
Aftergatheringandreprioritizingleftoverandnewtasks,anewBacklogisformedandanewSprintstarts.Potentially, many other organization and process patterns may be used in combination with the Scrumpatterns.
Figure:TheScrumpatternlanguage
THEPATTERNS
SPRINT
CONTEXT
You are a software developer or a coach managing a software development team where there is a highpercentageofdiscovery,creativity,ortestinginvolved.
Youarebuildingorexpandingsystems,whichallowpartitioningofwork,withcleaninterfacing,components,orobjects.
Scrum Master Sprint Backlog Scrum Meetings Demo After Sprint
ãJeffSutherland1993-2021 225
PROBLEM
Youwant to balance the needs of developers towork undisturbed and the need ofmanagement and thecustomertoseerealprogress,aswellascontrolthedirectionofthatprogressthroughouttheproject.
FORCES
• Developersneedtimetoworkundisturbed,buttheyneedsupportforlogistics;managementandusersneedtobeconvincedthatrealprogressisbeingmade.
• Often,bythetimesystemsaredelivered,theyareobsoleteorrequiremajorchanges.Theproblemisthatinputfromtheenvironmentiscollectedmostlyatthestartoftheproject,whiletheuserlearnsmostlybyusingthesystemorintermediatereleases.
• Itisoftenassumedthatthedevelopmentprocessisawell-understoodapproachthatcanbeplannedandestimated. Ifaproject fails, that isconsideredproofthatthedevelopmentprocessneedsmorerigor. These step by step approaches, however, don’t work because they do not cope with theunpredictabilties,bothhumanandtechnical,insystemdevelopment.Therefore,atthebeginningofaproject it is impossible tomakeacomplete,detailedspecification,plan,orschedulebecauseof themanyuncertaintiesinvolved.
• Overhead is often created to prove that a process is on track. Current process automation addsadministrativeworkformanagersanddevelopersandoftenresultsinmarginallyuseddevelopmentprocessesthatbecomedisk-ware.(Misfit:Activityisnotsynonymouswithresults.Moreoftenthatnot,aprojectplanshowsactivitiesbutfailstoensurerealprogressorresults.
SOLUTION
DividetheprojectinSprints.ASprintisaperiodofapproximately30daysinwhichanagreedamountofworkwillbeperformedtocreateadeliverable.EachSprinttakesapre-allocatedamountofworkfromtheBacklog,anditisassignedtoSprintsbypriorityandbyapproximationofwhatcanbeaccomplishedduringtheSprint’slength. In general, chunks of high cohesion and low coupling are selected—either horizontal or vertical“packets,”thatis,verticalorhorizontalcomponents.
ãJeffSutherland1993-2021 226
Asarule,nothingisaddedexternallytotheallocatedSprintBacklogduringtheSprint.ExternaladditionsareonlyaddedtotheglobalBacklog,butblocks(unresolvedissues)resultingfromtheSprintcanbeaddedtotheallocatedSprintBacklog.ASprintendwithademonstration(DemoAfterSprint)ofnewfunctionality.
Thisgivesthedevelopersspacetobecreative,tolearnbyexploringthedesignspaceandbydoingactualwork.Undisturbedbyoutsideinterruptions,theyarefreetoadapttheirwaysofworkingusingopportunitiesandinsights.Atthesametime,thiskeepsmanagementandotherprojectstakeholdersconfidentbyshowingrealprogressinsteadofdocumentsandreportsproducedasproofofprogress.
Thenetresultisthateachsprintproducesavisibleandusabledeliverablethatisshowntotheusersatthedemo(DemoAfterSprint).Anincrementcanbeeitherintermediateofshippable,butitshouldstandonitsown.ThegoalofaSprintistocompleteasmuchqualitysoftwareaspossibleandtoensurerealprogress,notpapermilestonesasalibis.
EditorsNote:Thisstrategyhadahugeimpactonglobalsoftwaredevelopment.Iterationsthatdemonstrateearlyworkingsoftwareinordertoincorporatereal-timeuserfeedbackhaveincreasedprojectsuccessindustry-widefrom16.2%in1994to35%in2006.Thisincreasedtheindustry-widereturnondollarinvestedinsoftwarefrom25centsin1998to59centsin2006foracompoundannualgrowthrateof24%[108].
RATIONALE
• ThefactthatnoitemsareaddedtotheBacklogexternallyallowsdevelopmenttoprogress“fullspeedahead,”withoutneedingtothinkaboutchangesindirection.
• Thefactthatdevelopersarenot“tested”duringtheSprintisempowering.
• The ability to choose a process per Sprint is empowering and enables adaptation to changingcircumstances(differentdevelopers,differentprojectphases,moreknowledge,etc.)
• Sprintsareshort;therefore,theproblemofcompletingaSprintismuchsimplerthatthatofcompletingaproject.Itiseasiertotakeupthissmallerchallenge.
• Developersgetfeedbackfrequently(attheendofeachSprint).Theycanthereforefeeltheirsuccesses(andfailures)withoutcompromisingthewholeproject.
• Managementhasfullcontrol—itcancompletelychangedirectionattheendofeachSprint.
ãJeffSutherland1993-2021 227
• TheendusersaredeeplyinvolvedthroughoutthedevelopmentoftheapplicationthroughtheDemosaftertheSprints,buttheyarenotallowedtointerferewiththeday-to-dayactivities.Thusownershipanddirectionstillbelongtotheusersbutwithouttheirconstantinterference.
• ProjectstatusisvisiblesincetheSprintproducesworkingcode.
KNOWN USES
At Argo, the Flemish department of education,we have been using Sprints since January 1997 on a largenumberofend-userprojectsandforthedevelopmentofaframeworkfordatabase,documentmanagement,andworkflow.TheBacklogisdividedinsprintsthatlastaboutamonth.AttheendofeachSprint,aworkingSmalltalk image is delivered with integration of all current applications. The team meets daily in ScrumMeetings,andBacklogisre-prioritizedaftertheDemoinamonthlymeetingwiththesteeringcommittee.
RESULTING CONTEXT
The result is ahighdegreeof “effectiveownership”by theparticipants, includinguserswhostay involvedthrough the Demos and the prioritizing of the Backlog. “Effective ownership” in this case means bothempowermentandtheinvolvementofalltheparticipants.
AttheendofaSprint,wehavethebestapproximationofwhatwasplannedatthestart.Inareviewsession,thesupervisorshavetheopportunitytochangetheplanningforthefuture.Theprojectistotallyflexibleatthispoint.Target,product,deliverydate,andcostcanberedefined.
WithScrumwegetalargeamountofpost-planningflexibility(forbothcustomeranddeveloper).
ItmaybecomeclearinthedailyScrumMeetingsthroughouttheSprintthatsometeammembersarelosingtimeatnon-orlessproductivetasks.Alternatively,itmayalsobecomeclearthatpeopleneedmoretimefortheir tasks than originally allocated by management. Developers may turn out to be less competent orexperiencedattheallocatedtaskthanassumed,ortheymaybeinvolvedinpoliticalorpowerstruggles.Thehighvisibilityof scrum,however, allowsus todealwith theseproblems.This is the strengthof theScrummethodmanifestedthroughtheScrumMeetingsandtheSprints.
ãJeffSutherland1993-2021 228
DifficultiesingroupingBacklogforaSprintmayindicatethatprioritiesarenotcleartomanagementortothecustomer.
BACKLOG
CONTEXT (FROM: SPRINT)
Youareconnectedtoasoftwareprojectoranyotherprojectthatischaoticinnaturethatneedsinformationonwhattodonext.
PROBLEM
Whatisthebestwaytoorganizetheworktobedonenextandatanystageoftheproject?
FORCES
Traditional planningmethods likePert andGantt assume that youknow in advance all the tasks, all theirdependencies, all task durations, and all available resources. These assumptions are wrong if the projectinvolvesanylearning,discovery,creativity,oradaptation.
SOLUTION
UseaBacklogtoorganizetheworkofaScrumteam.
TheBacklogisaprioritizedlist.ThehighestpriorityBacklogitemwillbeworkedonfirst,thelowestpriorityBacklogitemwillbeworkedonlast.Nofeature,addition,orenhancementtoaproductisworthfightingover;itissimplyeithermoreimportantorlessimportantatanytimetothesuccessandrelevanceoftheproduct.
Backlogistheworktobeperformedonaproduct.Completionoftheworkwilltransformtheproductfromitscurrentformintoitsvision.ButinScrum,theBacklogevolvesastheproductandtheenvironmentinwhichit
ãJeffSutherland1993-2021 229
willbeusedevolves.TheBacklogisdynamic,constantlychangedbymanagementtoensurethattheproductdefinedbycompletingtheBacklogisthemostappropriate,competitive,usefulproductpossible.
TherearemanysourcesfortheBackloglist.Productmarketingaddsworkthatwillfulfilltheirvisionoftheproduct.Salesaddsworkthatwillgeneratenewsalesorextendtheusefulnesstotheinstalledbase.Technologyaddsworkthatwillensuretheproductusesthemostinnovativeandproductivetechnology.Developmentaddsworktoenhanceproductfunctions.Customersupportaddsworktocorrectunderlyingproductdefects.
Onlyonepersonprioritizeswork.Thispersonisresponsibleformeetingtheproductvision.Thetitleusuallyisproductmanagerorproductmarketingmanager.Ifanyonewantsthepriorityforworkchanged,theyhavetoconvincethispersontochangethatpriority.ThehighestpriorityBackloghasthemostdefinition. It isalsoprioritizedwithaneyetowarddependencies.
Dependingonhowquicklyproductsareneededinthemarketplaceandthefinancesoftheorganization,oneormoreScrumTeamsworkonaproduct’sBacklog.AsaScrumTeamisavailable(newlyformedorjustfinishedaSprint)toworkontheBacklog,theteammeetswiththeproductmanager.FocusingonthehighestpriorityBacklog,theteamselectsasubsetoftheBacklogtheteambelievesitcancompletewithinaSprintiteration(30daysorless).Indoingso,theScrumTeammayaltertheBacklogprioritybyselectingaBacklogthatismutuallysupportive,thatis,onethatcanbeworkedonatoncemoreeasilythanbywaiting.ExamplesaremultipleworkitemsthatrequiredevelopingacommonmoduleorinterfaceandthatmakesensetoincludeinoneSprint.
TheteamselectsacohesivegroupoftoppriorityBacklogitemsthat,oncecompleted,willhavereachedanobjective,ormilestone.ThisisstatedastheSprint’sobjective.DuringtheSprint,theteamisfreetonotdoworkaslongasthisobjectiveisreached.
TheteamnowdecomposestheselectedBacklogintotasks.Thesetasksarediscretepiecesofworkthatvariousteammemberssignuptodo.TasksareperformedtocompleteBacklogtoreachtheSprintobjective.
RESULTING CONTEXT
Projecworkisidentifieddynamicallyandprioritizedaccordingto:
1. Thecustomerneeds
ãJeffSutherland1993-2021 230
2. Whattheteamcando
SCRUMMEETINGS
CONTEXT (FROM: BACKLOG)
You are a software developer or a coach managing a software development team where there is a highpercentageofdiscovery,creativity,ortestinginvolved.Anexampleisafirsttimedeliverywheretheproblemhastobespecified,anobjectmodelhastobecreated,orneworchangingtechnologiesarebeingused.
Activitiessuchasscientificresearch, innovation, invention,architecture,engineeringandamyriadofotherbusinesssituationsmayalsoexhibitthisbehavior.
Youmayalsobea“knowledgeworker,”anengineer,awriter,aresearchscientist,oranartist,oracoachormanagerwhoisoverseeingtheactivitiesofateamintheseenvironments.
PROBLEM
What is the best way to control an empirical and unpredictable process such as software development,scientificresearch,artisticprojects,orinnovativedesignswhereitishardtodefinetheartifactstobeproducedandtheprocessestoachievethem?
FORCES
ESTIMATION
• Accurate estimation for activities involving discovery, creativity, or testing is difficult because ittypically involves large variances, and because small differences in circumstances may causesignificantdifferencesinresults.Theseuncertaintiescomeinatleastfiveflavors:
1. Requirementsarenotwellunderstood.
ãJeffSutherland1993-2021 231
2. Architecturaldependenciesarenoteasytounderstandandareconstantlychanging.
3. Theremay be unforeseen challengeswith the technology. Even if the challenges are know inadvance,theirsolutionsandrelatedeffortarenotknown.
4. Theremaybebugsthatarehardtoresolveinthesoftware;therefore,itistypicaltoseeprojectestimatesthatareseveralordersofmagnitudeoff.Youcan’t“planbugs,”youcanonlyplanbughandling and provide appropriate prevention schemes based on the possibility of unexpectedbugs.
Example:YouGottheWrongNumber.Inprojectswithneworchangingrequirements,anewarchitecture,neworchangingtechnologies,anddifficultbugstoweedout,itistypicaltoseeprojectestimatesthatareoffbyseveralordersofmagnitude.
5. Ontheotherhand,estimationis important.Onemustbeabletodeterminewhatarethefuturetaskswithinsometimehorizonandprepareresourcesinadvance.
PLANNING
• Planningandreprioritizingtaskstakestime.Involvingworkersintimeplanningmeetingsdecreasesproductivity.Moreover,ifthesystemischaotic,noamountofplanningcanproduceuncertainties.
Example: Paralysis by Planning. Some projects that waste everyone’s time in planningeverythingtoanextremedetailbutareneverabletomeettheplans.
• Aplanthatistoodetailedbecomelargeandishardtofollow;thelargertheplanis,themoreerrorsitwillcontain(orattheveryleastthecostofverifyingitscorrectnessgrows).
Example:TheMasterPlanIsaGreatBigLie.Manyprojectsthattrytofollowamasterplanfallintothetrapofactuallybelievingtheirinaccuraciesandoftenfacedisappointmentwhentheirexpectationsarenotmet.
• Noplanningatallincreasesuncertaintyamongteammembersandeventuallydamagesmorale.
Example:LostVision.Projectsthatneverscheduleanythingtendto losecontrolovertheirexpectations.Without some schedule pressure no onewill do anything, andworse, itwillbecomedifficulttointegratethedifferentpartsbeingworkedonindependently.
TRACKING
• Toomuchmonitoringwastestimeandsuffocatesdevelopers.
ãJeffSutherland1993-2021 232
Example:MeasuredtoDeath.Projectsthatwasteeverybody’stimeintrackingeverythingtoanextremedetailbutareneverabletomeettheplans.(Youmeasuredthetirepressureuntilalltheairwasout!)
• Trackingdoesnotincreasethecertaintyofindicatorsbecauseofthechaoticnatureofthesystem.Infact, trying to control normal variations of a system will cause wide oscillations of the system,renderingitmorechaotic.
• Toomuchdataismeaningless—theNeedleintheHaystackSyndrome.
• Notenoughmonitoringleadstoblocksandpossibleidletimebetweenassignments.
Example:WhatHappenedHere?Projectsthatnevertrackanythingtendtolosecontroloverwhatisbeingdone.Eventuallynoonereallyknowswhathasbeendone.
SOLUTION Toprovideforaccurateestimates,plans,andappropriatetracking,meetwiththeteammembersforashorttime(~15minutes)inadailyScrumMeeting,wheretheonlyactivityisaskingeachparticipantthefollowingthreequestions:
1. WhathaveyouworkedonsincethelastScrumMeeting?TheScrumMasterlogsthetasksthathavebeencompletedandthosethatremainundone.
2. Whatblocks,ifany,haveyoufoundinperformingyourtaskswithinthelast24hours?TheScrumMasterlogsallblocksandlaterfindsawaytoresolvetheblocks.
3. Whatwillyoubeworkingoninthenext24hours?TheScrumMasterhelpstheteammemberschoose theappropriate tasks toworkonwith thehelpof theArchitect.Because the tasks arescheduledona24-hourbasis,thetasksaretypicallysmall(SmallAssignments).
Thiswill provide youwithmore accurate estimates, short-termplans, appropriate tracking, an correctingmechanismstoreacttochangesandadaptevery24hours.
ScrumMeetingstypicallytakeplaceatthesametimeandplaceeveryday,sotheyalsoservetobuildastrongculture.Assuch,Scrummeetingsareritualsthatenhancethesocializationofstatus,issues,andplansfortheteam.TheScrumMasterleadsthemeetingsandlogsallthetasksfromeverymemberoftheteamintoaglobalproject Backlog. He also logs every block and resolves each block while the developers work on otherassignments.
Editorsnote:TheScrumBoardhasemergedasabestpracticeforateamtomanagetheirowntasks.Teamsmeetin frontoftheBoardwhichhasmultiplecolumns.ThefirstcolumnhasUserStories fromtheProductBacklog(featurestobedelivered)onlargecardsprioritizedinorderofbusinessvalue.AtthestartoftheSprint,thetaskstobeaccomplishedforaUserStoryareintheleftcolumnassmallcards.Eachdaydevelopersmovetaskstoan“InProgress”column,thentoa“Validation”column,thentoa“Done”column.EstimatesareupdatedontasksdailyandtheBurndownChartcaneasilybecalculatedandpostedtotheboard[109].
ãJeffSutherland1993-2021 233
TheblocksloggedbytheScrumMasterarenowknownasthe“ImpedimentList”whichneedstobeprioritized.TheblockwhichisthemostcriticalconstrainttosystemthroughputshouldbeatthetopofthelistandtheScrumMastershouldworkonthatonefirst.Tuningadevelopmentprojectissimilartotuningacomputersystem.Itmaynotbeobviouswherethecriticalconstraintliesandcarefulanalysismayberequired.Themainchokepointmustbefoundandfixedfirst.Thedevelopmentsystemasawholeshouldthenbeallowedtostabilizeandmeasured.Thenextcriticalblockafterrestabilizationmaybeinanunexpectedplace.Thatshouldbefixednext.Fixingtoomanythingsatoncegenerateswastebyfixingconstraintsthathaveminimalimpactonthroughput.Thisusescriticalresources to change things that do not dramatically improve velocity. It makes it difficult to clarify systemdynamicsandtiresoutanddemotivatestheteam,management,andthecompany.
Scrummeetingsnotonlyscheduletasksfordevelopers,butcanandshouldscheduleactivitiesforeveryoneinvolved in the project, such as integration personnel dedicated to configurationmanagement, architects,ScrumMasters,oraQAteam.
ScrumMeetingsallowknowledgeworkerstoaccomplishmid-termgoalstypicallyallocatedinSprintsthatlastamonthorless.
ScrumMeetingscanalsobeheldbyself-directedteams.Inthatcase,someoneisdesignatedasthescribeandlogs thecompletedactivitiesof theBacklogand theexistingblocks.Allactivities fromtheBacklogand theblocksandthendistributedamongtheteamforresolution.
Theformatof theBacklogandtheblockscanalsovary,rangingfroma listof itemsonapieceofpaper, tosoftwarerepresentationsofitovertheInternet/Intranet[18].TheScrumMeeting’sfrequencycanbeadjustedandtypicallyrangesbetween2and48hours.
Thesemeetingsareoftenheldstandingup.Thisensuresthatthemeetingsarekeptshortandtothepoint.
ãJeffSutherland1993-2021 234
RATIONALE Itisveryeasytoover-orunder-estimate,whichleadseithertoidledevelopertimeortodelaysincompletionofanassignment.Therefore,itisbettertofrequentlysamplethestatusofsmallassignments.ProjectswithahighdegreeofunpredictabilitycannotusetraditionalprojectplanningtechniquessuchasGanttorPERTchartsonly, because the rate of change of what is being analyzed, accomplished, or created is too high. Instead,constantreprioritizationoftasksoffersanadaptivemechanismthatprovidessamplingofsystemicknowledgeovershortperiodsoftime.
ScrumMeetings help also in the creation of an “anticipating culture” [104] because they encourage theseproductivevalues:
• Theyincreasetheoverallsenseofurgency.
• Theypromotethesharingofknowledge.
• Theyencouragedensecommunications.
• Theyfacilitatehonestyamongdeveloperssinceeveryonehastogiveadailystatus.
Thissamemechanismencouragesteammemberstosocialize,externalize,internalize,andcombinetechnicalknowledge on an ongoing basis, thus allowing technical expertise to become community property for thecommunityofpractice[110].ScrumMeetingsarethereforeritualswithdeepculturaltranscendence.Meetingatthesameplaceatthesametime,andwiththesamepeople,enhancesafeelingofbelongingandcreatesthehabitofsharingknowledge.
SeenfromtheSystemDynamicspointofview[88],softwaredevelopmenthasaschedulingproblembecausethenatureofprogrammingassignmentsisratherprobabilistic.Estimatesarehardtocomebybecause:
• Inexperienceddevelopers,managers,andarchitectsareinvolvedinmakingtheestimates.
• Therearetypicallyinterlockingarchitecturaldependenciesthatarehardtomanage.
• Thereareunknownorpoorlydocumentedrequirements.
• Thereareunforeseentechnicalchallenges.
Asaconsequence,thesoftwaredevelopmentbecomesachaoticbeergame,whereitishardtoestimateandcontrol the inventory of available developer’s time, unless increased monitoring of small assignments isimplemented [88, 111]. In that sense the ScrumMeeting becomes the equivalent of the thermometer thatconstantlysamplestheteam’stemperature.
FromtheComplexityTheoryperspective[105,106],Scrumallowsflockingbyforcingafasteragentinteraction,thereforeacceleratingtheprocessofself-organizationbecauseitshiftsresourcesopportunisticallythroughthedailyScrumMeetings.
ãJeffSutherland1993-2021 235
Thisisunderstandable,becausetherelaxationoftheself-organizedmulti-agentsystemisproportionaltotheaverageexchangeamongagentsperunitoftime.Andinfact,the“interactionrate”isoneoftheleversonecanpushtocontrol“emergent”behavior—itislikeaddinganenzymeorcatalysttoachemicalreaction.
In Scrum this means increasings the frequency of the Scrum Meetings, and allowing more hyperlinks asdescribedbelow,butuptoanoptimalupper-frequencyboundontheScrumMeetings(meetings/time),anduptoanoptimalupperboundonthehyperlinksortheScrumTeammembers.Otherwisetheorganizationspendstoomuchtimesocializingknowledge,insteadofperformingtasks.
KNOWN USES (MikeBeedle)AtNikeSecuritiesinChicagowehavebeenusingScrumMeetingssinceFebruary1997torunallofourprojectsincludingBPRandsoftwaredevelopment.EveryoneinvolvedintheseprojectsreceivesaweekoftraininginScrumtechniques.
(YonatSharon)AtElementrixTechnologieswehadaprojectthatwasrunningwaylateafteraboutfivemonthsofdevelopment.Onlyasmallpart(about20percent)wascompleted,andeventhisparthadtoomanybugs.Theprojectmanagerstartedrunningbi-dailyshortstatusmeetings(noneofuswas familiarwiththetermScrumbackthen).Inthefollowingmonth,theentireprojectwascompletedandthequalityhadrisensharply.Twoweekslater,abetaversionwasout.Themeetingswerediscontinued,andtheprojecthardlyprogressedsince.Idon’tthinkthesuccessoftheprojectcanbeattributedtotheScrumMeetingsalone,buttheydidhaveabigpartinthisachievement.
OneofmysoftwareteamleadersatRafaelimplementedavariationofScrumMeetings.Hewouldvisiteachdeveloperonceaday,andaskhimthethreequestions;healsomanagedaBacklog.Thisdoesnothavetheteambuildingeffects,butitdoesprovidethefrequentsampling.
C3andVcapsprojects(describedonwiki[112])alsodothis.(BTW,IadoptedthisnameinHebrew,sinceinHebrew“meeting”is“sitting,”andsowesay“standupsitting”.)
RESULTING CONTEXT Theapplicationofthispatternleadsto:
• Highlyvisibleprojectstatus
• Highlyvisibleindividualproductivity
• Lesstimewastedbecauseofblocks
• Lesstimewastedbecauseofwaitingforsomeoneelse
• Increasedteamsocialization
CONCLUSIONScrumisaknowledgecreatingprocesswithahighlevelof informationsharingduringthewholecycleandworkprogress.
ãJeffSutherland1993-2021 236
ThekeytoScrumispinningdownthedateatwhichwewantcompletionforproductionorrelease,prioritizingfunctionality, identifying available resources, andmakingmajordecisions about architecture.Compared tomore traditionalmethodologies, the planning phase is kept short sincewe know that eventswill requirechangestoinitialplansandmethods.Scrumusesanempiricalapproachtodevelopmentwhereinteractionwiththe environment is not only allowed but encouraged. Changing scope, technology, and functionality areexpected;andcontinuousinformationsharingandfeedbackkeepsperformanceandtrusthigh.
Itsapplicationalsogeneratesastrongculturewithwell-definedrolesandrelationships,withmeaningfulandtranscendingrituals.
ACKNOWLEDGEMENTSWewouldliketothankalloftheScrumusersandreviewersfromwhomwehavereceivedfeedbackovertheyears.Also,wethankallofthemembersoftheChicagoPatternsGroupthatattendedanearlyreviewsessionof the ScrumMeetingpattern (especiallyBradAppleton, Joe Seda, andBobHaugen). Finallywe thankourPloP’98shepherd,Lindarising,forprovidinguscommentsandguidancetomakeourpaperbetter.
(PersonalacknowledgementfromMikeBeedle.)I’dliketothankbothJeffSutherland[113]andKenSchwaber[114]foradaptingtheScrumtechniquestosoftwareintheearly1990s,andforsharingtheirfindingswithme.ScrumhasmadeasignificantcontributiontothesoftwareprojectsinwhichIusedthetechnique.
ãJeffSutherland1993-2021 237
INDEX
A
AgileManifesto·9,205
B
BorlandQuattroPro·5,164businessplan·183
C
CapersJones·6,109ChiefEngineer·9CMMI·8,9,82,97,172,204
E
EarnedBusinessValue·155,158,159,160,161,162
F
Fuji-Xerox·46,56,93,168
G
Ganttcharts·159,195,196,197
H
HitotsubashiBusinessSchool·8Honda·8,46,48,56,86,93,95,168
J
JeffMcKenna·6,85,167JimCoplien·5JohnScumniotales·6,50,167
K
KentBeck·9,54
M
microenterprisedevelopment·5
O
ObjectStudio·9OOPSLA’95·9,54
P
PasteurProject·5PatientKeeper·9,47,54,83,85,91,92,109,166,167,176,177,181,182,183,184,185,186,187,190,193,194,195,196,197,200
ProductBacklog·9,49,50,81,83,155,161,162,167,174,175,179
ProductOwner·6,8,9,50,81,96,102,103,104,106,162,167,168,170,171,175,177,178,179,180,181,182,183,184,193,197,210
Product Specifications·104
Q
QualityAssurance·176,190
R
RodneyBrooks·6
S
ScrumandXP·9,104
ãJeffSutherland1993-2021 238
ScrumMaster·6,9,50,54,81,83,102,167,170,172,179,181,183,184,187,189,204
SirsiDynix·8,81,93,96,97,98,99,100,101,102,103,104,107,108,109,110,164,204
SprintBacklog·9,49,83,109,155,174,179,182,200SprintPlanning·107,169StarSoft·81,93,98,99,100,101,102,108,109,204subsumptionarchitecture·6,95
T
TakeuchiandNonaka·5,8,48,56,95,168,170,175,176
Toyota·8,46,49,50,83,89,93,96,167,169,170,171,172,173
W
Waterfall·59,60,62,64,65,109,164
Top Related