The Magazine for Agile Developers and Agile Testers · “How can you automate stuff when you su-...
Transcript of The Magazine for Agile Developers and Agile Testers · “How can you automate stuff when you su-...
The Magazine for Agile Developers and Agile Testers
© iStockphoto.com/ThomasVogel
January 2011
issue 5www.agilerecord.com freedigitalversion madeinGermany ISSN2191-1320
77www.agilerecord.com
Thefirstfewthingsthatcometoourmindwhenwetalkaboutautomationarerepeatability,reducedtimeandcosts,reusability,reliability andbetter quality of softwarebeingdeveloped.Withthe increasing use of the Agile developmentmethodology (in-crementaland iterativemodelofdevelopment) inthesoftwareindustry,theroleandbenefitsofautomationinAgilehasalwaysbeenunderscrutiny.“Howcanyouautomatestuffwhenyousu-relyknowthatthefeaturebeingdevelopedissubjecttochangein thenext iteration?” isabigquestionposedbyqualityassu-ranceteams.
The objective of this article is to bring out the risks and chal-lengestheproductteamsfacewhileautomatingtestsusinganAgiledevelopmentmethodology.TheauthorsalsosuggestsomestrategiesthatproductteamscanadoptsothatautomationcanbeeffectivelydevelopedandusedinanAgilecontext.
How Agile works?AccordingtoWikipedia,“Agile software development is a group of software development methodologies based on iterative and incremental development, where requirements and solu-tions evolve through collaboration between self-organizing, cross-functional teams.” Agile advocates frequent releases inshortdevelopmentcycles.InAgile,thedevelopmentofsoftwareis broken down into smaller tasks based on product features.Thecompletesoftwaredevelopmentlifecycle(requirementana-lysis,design,developmentandtesting)isfollowedforeachtask.Thetaskistargetedtobecompletedinaperiodof2-4weeks(atypicaldurationofdevelopmentcycle).Whenthetaskiscomple-tedthenextdevelopmentiterationbeginswherenewandincre-mentaltasksaretakenupbytheproductteams.Theobjectiveofadevelopmentcycleisnottodevelopsoftwareofproductionqualitybut tohaveearlyand frequent feedbackson the tasks(oronthedevelopedfeatures)fromthecustomers.Thisnotonlyhelpstheproductteamstoreducerisksbutalsohelpsthemtobeflexibletochanges.
NowthatweunderstandhowAgileworks,let’sanalyzethechal-lengesQAteamsfacewhiledevelopingautomationandtherisksassociatedwithautomationinanAgileset-up.
Challenges and risks of automationTestautomationispreferablydoneinsituationswhere:
• thetasksarehighlyredundant.
• thesoftwareistestedmanytimesandisquitestable.
• thepossibilityoffeaturegettingchangedisminimal.
• thetasksaresorepetitivethatavoidinghumanerrorisad-vantageous.
WithAgile,thesoftwaregetsdevelopedincrementallyin‘N’de-velopmentcycles.Planninganddevelopingautomatedtestsforafeaturebecomesdifficultinthisdevelopmentmode.
ListedbelowarethechallengesandrisksofautomationinAgile.
My basis for automation is no longer valid!InAgile,thereisapossibilityoffeaturesgettingremovedduringcustomerinteractionsattheendofeverydevelopmentcycle.Youmighthavespenttimeautomatingtests fora featureofawebapplicationandinthenextcycleyourealizethefeaturenomoreexistsorhasbeendrasticallychanged.Insuchsituations,eitheralotofreworkisrequiredonthescriptsortheteststhatyouhaveautomatedmaynolongerbeusablefortestingthewebapplica-tion.ThisresultsinwastageofeffortonthepartofQAteams.
Maintaining the automation is costlyInAgilemethodology,thesoftwaregetsdevelopedincrementally.Asmallchangeinthewebpageofawebapplicationoramodifi-cationinGraphicalUserInterfaceofanapplicationwouldmeanspending time inmodifying thealreadydeveloped test scripts.Itprovescostlyfortheautomationengineerstoinvesttimeandenergymaintainingthesameautomationscript(s)witheveryde-velopmentcycle.
© iStockphoto.com
/nullplus
Automation in Agileby Chetan Giridhar & Sunil Ubranimath
78 www.agilerecord.com
Test Bed MaintenanceTheessentialrequirementsfortesting(softwareorhardware)fallunderthecategoryof‘testbed’.Forachangeinthesoftwarede-signorthecode,aneedalsoarisesforchangingthetestbed.Forexample,additionofafeaturecanleadtoincreaseintestdatafiles,maintenanceofwhichgetsdifficultattimes.
Automation involves time and planningThisisoneoftherisksassociatedwithautomationinAgile.Auto-mationessentiallymeansdevelopingoftestscriptswhichcouldhelpinreducingexecutiontimesandcosts.Buttheflipsideis,writing test scripts requiresagoodamountofplanning,whichtakes time. Agile development cycles are typically 2-4 weeks,whichdoesn’tallowforsufficientplanningforautomation.
On what scale do I design my automation?Agoodautomationframeworkisonewhichhasarobustarchitec-tureand library. InAgileenvironments it’sdifficult forautoma-tionengineerstoworkonthearchitectureoractuallybuildtheframework,asthearchitectureorthedesigntheydevelopedinprevious development cyclesmay not be suitable for creatingandrunningautomatedtestsfornewfeaturesbeingdeveloped.ThereisalwaysanuncertaintyonwhatscaletheautomationcanbedesignedorbuiltinAgile.
Automation teams different from QA teamsIt iscommonpractice forproductcompanies tohaveseparateautomation and QA teams because QA engineersmay not beskilledinwritingtestscriptsorbuildingframeworks.Theworkofanautomationengineerisoftencloselyrelatedtobothdevelop-mentandQA.Theautomationengineershouldhaveanattitudeof breaking the codeandat the same time shouldbeable towrite codeasdevelopersdo.Withadifferent setof engineersdeveloping the test scripts there is the advantage of having adifferentperspectivefortestingthesamesoftware.However,thedisadvantageofthisapproachisthatautomationengineershavetoconstantlycommunicatewithdevelopmentandQA,whichme-ansmoretimespentbytheseteamswithautomationengineers.ThisisariskwhenitcomestoanAgilecontextasthereisalwaysashortageoftimeinAgile.Moreover,automationengineersaresobusyworkingonthedevelopmentofscriptsandlibrariesthattheygetinsufficienttimetogainproductknowledge,whichdef-eatsthepurposeoftesting.
Change in focus on the part of automation engineers Oftenautomationengineersgetoverinvolvedinwriting“great”code.Engineersgetbusyinwritingtestscripts,buildinglibrariesorframeworksthatmaybeefficient,butmaynothelpintestingthe softwaremore effectively. In Agile environments, the con-sequences of such situations are fatal as the amount of timeavailable forcreatinggoodcode is reduced.This isoneof therisksassociatedwithautomationinAgile.
Product testabilityOneofthenon-functionalcharacteristicsofsoftwareistestabili-ty.AccordingtoWikipedia,“Softwaretestabilityisthedegreetowhichasoftwareartifact(i.e.asoftwaresystem,softwaremodu-
le,requirements-ordesigndocument)supportstestinginagiventestcontext.”Alowerdegreeoftestabilityoftenimpliesmoretes-tingeffort.Oneofthefactorsonwhichtestabilityisdependentisautomatability(thedegreetowhichasoftwareundertestcanbesubjectedtoautomatedtesting). Ifawebapplicationrespondstouserrequestsin,say,0.2seconds,thisisanacceptableper-formancemetric,butifitisnotopeningwithinatimeperiodof1-2minutes,thewebapplicationcanbedeemednon-testable.InAgiletheproductisdevelopedinanincrementalfashionandperformance(eventhoughverycrucial)isNOTconsideredatthedesignphaseof theproduct.Suchproductsmaywork functio-nallycorrectbutiftheperformanceisnotgood,testingofsuchapplicationsbecomesdifficult. Investing time in automationofnontestableproductswouldbearisk.
Choice of Automation Tools is importantWeunderstand thatautomation in theearly stagesofproductdevelopmentinAgileisdifficult.Butasmoredevelopmentcyclesarecompleted,thefeaturesgetdevelopedandthesystembeco-messtable.Itisthusverycrucialtoselectagoodtestingtoolforautomationthatcanbeusedthroughouttheproductlifecycle.
CommunicationCommunicationplaysavitalroleinanAgilecontext.Itisimpor-tantforautomationengineerstocommunicatewellwithQA,de-velopmentteams,aswellaswiththeotherstakeholdersoftheprojecttocorrectlyunderstandtherequirements.Workingincol-laborationandcommunicatingwellusingdifferentchannels(likeScrummeetings)isthekeytosuccess.
Suggested automation strategies for AgileAutomationispossibleandshouldbedoneinAgile,butthefol-lowingpointsmustbekeptinmind.
Quick GainsConsiderautomationfor featuresthatcanprovidequickgains.Anysmalltask,iffoundrepeatableandvalidthroughoutthepro-ductlifecycle,wouldbeconsideredasagoodcandidateforauto-mation.Notonlydoesittakelesstimeforbuildingtheautomati-on,butitisalsoadvantageousasthesametasksareperformedforeverydevelopmentcycle.
Loosely coupled automation frameworkA very generic test automation frameworkwhere tests can becontributed easily for the new features in development cycleswouldbethekeytoautomationinAgile.ItbecomesdifficultforQAandautomationengineerstocontributetestsinahard-codedframework. A very loosely coupled framework that is indepen-dentoftheproductortheproductfeaturesaccommodatestestsfornewfeaturesfairlyeasily.
Building on the source code and unit testsItmaybeworthwhileforautomationengineerstohavealookatthesourcecodeandtheunittestsandreusethesameforbuil-dingtheirautomatedtests.Changesdoneinthesourcecodeforaccommodatinganewfeatureorbuildingontheexistingfeaturecanbeeasilyreflectedinautomationbyfollowingthisapproach.
80 www.agilerecord.com
Chetan Giridharhas more than 5 years of experience working as a software engineer in re-search and product organi-zations. Chetan is an avid blogger and has a blogspot which he updates with ar-ticles, blogs and publica-tions. You can reach him at [email protected].
Sunil Ubranimathhas more than 7 years of experience in the software industry and has good ex-posure on Agile practices. He has worked in the Agile environment for more than 3 years. You can reach Sunil at [email protected]
> About the authorThisalsodevelops rapportbetweendevelopmentandautoma-tionengineersand inawaycodegetsreviewedbyautomationengineers.
Vertical DevelopmentDevelopmentofafeatureisdeemedtobecompleteonlywhenthe functional (alongwith robusterrorhandling)andnon-func-tional aspects of the feature are complete. Development canhappenintwodistinctways.Horizontaldevelopmentinvolvesde-velopmentoffunctionalitiesonly.Thiswouldmeanwhenallthefeaturesarefunctionallycorrect,errorhandlingandperformanceaspectswouldbeaddressedforallthefeatures.Contrarytothisapproach,verticaldevelopmentwouldmeandevelopingonefea-tureinclusiveofallfunctionalrequirements,robusterrorhand-lingandnon-functionalaspectsfortheparticularrequirement.InAgile,itisadvisabletofollowvertical(couldalsobereferredtoasdepth-wise)developmentbecausethiswouldhelpautomationengineersplanbetterforwritingthetestscriptsforfeaturequali-fication.Withverticaldevelopment,nonfunctionalrequirementswillbeconsideredearlywhichimprovesthetestabilityofthecom-ponentundertest.
ConclusionInthisarticleweintroducedtheAgiledevelopmentmethodology.We also considered the challenges ofwriting automated testsinanAgileset-upalongwiththerisksthatautomationcanposewhileworkinginAgile.Thearticlealsosuggestedsomepracticesthatautomationengineerscanfollowsothat theycandevelopanduseautomationwitheaseinAgile.■
References• www.wikipedi.org–thefreeencyclopedia
• Paperon‘AgileTestAutomation’
www.satisfice.com/articles/agileauto-paper.pdf
ByJamesBach
• Automated Testing in anAgileEnvironment–http://www.tampabayqa.com/Images/Agile.ppt
ByBobCrewsandKenArneson,Checkpoint TechnologiesInc.