IMPROVE UNIT TESTING FOR JAVA WITH AUTOMATION

BEST PRACTICES FOR JAVA DEVELOPERS

03 Introduction:ImproveUnitTesting 03 WhatIsUnitTesting?

03 ImproveUnitTestingWithAutomatedTestingTools

04 BestPracticesforDevelopers 04 UnitTestingBestPractices:HowtoGettheMost outofYourTestAutomation

10 JUnitTutorial:SettingUp,Writing,andRunningJavaUnitTests

17 MockinginJava:HowtoAutomateaJavaUnitTest, IncludingMockingandAssertions

24 HowtoCreateJUnitParameterizedTests

34 GetMoreOutofUnitTestingandReduceMaintenance EffortsWithRuntimeAnalysis

Table of Contents

Improve Unit Testing for Java With AutomationBest Practices for Java Developers

Introduction: Improve Unit TestingWHAT IS UNIT TESTING?Unittestingisthepracticeoftestingindividualunitsorcomponentsofanapplicationinordertovalidatethateachofthoseunitsisworkingproperly.Generally,aunitshouldbeasmallpartoftheapplication—inJavait'softenasingleclass.Notethatthereisnostrictdefinitionof"unit"here,anditisuptothedevelopertodecidethescopeoftestedcodeforeachtest.

Peoplesometimescontrasttheterm"unittesting"with"integrationtesting"or"end-to-endtesting".Thedifferenceisthat,generally,unittestingisdonetovalidatethebehaviorofanindividualtestableunit,whereasintegrationtestsarevalidatingthebehaviorofmultiplecomponentstogether,ortheapplicationasawhole.Asmentionedabove,thedefinitionforwhatconstitutesa"unit"isnotstrictlydefined,andit'suptoyoutodecidethescopeforeachtest.Ifthescopeistoobroad,itmaynotbepossibletodeterminewhyatestfailureoccurred.

Withthesechallenges,unittestingjustisn'teasy.Itrequiresalotofdevelopmentskillandeffort,andittakescommitmentandtimetomaintaintestsuites.ThisebookprovideshelpfultipsandtechniquesaswellasbestpracticestohelpyouimproveunittestingwithJUnitandParasoftJtest.

IMPROVE UNIT TESTING WITH AUTOMATED TESTING TOOLS Withautomatedtestingtools,developersareabletoreducelate-cycledefectswithbetterunittestsandautomatedstaticcodeanalysis.Theycanfocusmoretimeonnewfeaturedevelopmentforthebusiness.Developersalsobenefitfromimmediatefeedback.They'reabletorapidlyidentifywhethertheircodechangesarebreakingfunctionalityintheapplicationandaddressingitquickly.

Parasofthasfocusedonimprovingautomatedtestingforover30years.ParasoftJtestisakeyenablerofdeliveringqualityatspeedforunittesting.ThisintegratedJavasolutionenablesdevelopmentteamstobeagileanddeliverfasterwithoutsacrificingquality,makingthebusinesssuccessful.

Best Practices for DevelopersUNIT TESTING BEST PRACTICES: HOW TO GET THE MOST OUT OF YOUR TEST AUTOMATION Unittestingisawell-knownpractice,butthere'slotsofroomforimprovement!Thissectionwillcoverthemosteffectiveunittestingbestpractices,includingapproachesformaximizingyourautomationtoolsalongtheway.Itwillalsodiscusscodecoverage,mockingdependencies,andoveralltestingstrategies.

WHYUNITTEST? Unittestingisaproventechniqueforensuringsoftwarequality,withplentyofbenefits.Hereare(morethan)afewgreatreasonstounittest:

» Unittestingvalidatesthateachpieceofyoursoftwarenotonlyworksproperlytoday,butcontinuestoworkinthefuture,providingasolidfoundationforfuturedevelopment.

» Unittestingidentifiesdefectsatearlystagesoftheproductionprocess,whichreducesthecostsoffixingtheminlaterstagesofthedevelopmentcycle.

» Unit-testedcodeisgenerallysafertorefactor,sincetestscanbere-runquicklytovalidatethatbehaviorhasnotchanged.

» Writingunittestsforcesdeveloperstoconsiderhowwelltheproductioncodeisdesignedinordertomakeitsuitableforunittesting,andmakesdeveloperslookattheircodefromadifferentperspective,encouragingthemtoconsidercornercasesanderrorconditionsintheirimplementation.

» Includingunittestsinthecodereviewprocesscanrevealhowthemodifiedornewcodeissupposedtowork.Plus,reviewerscanconfirmwhetherthetestsaregoodonesornot.

It'sunfortunatethatalltoooften,developerseitherdon'twriteunittestsatall,don'twriteenoughtests,ortheydon'tmaintainthem.Unittestscansometimesbetrickytowrite,ortime-consumingtomaintain.Sometimesthere'sadeadlinetomeet,anditfeelslikewritingtestswillmakeusmissthatdeadline.Butnotwritingenoughunittestsornotwritinggoodunittestsisariskytraptofallinto.

Sopleaseconsiderthefollowingbest-practicerecommendationsonhowtowriteclean,maintainable,automatedteststhatgiveyouallthebenefitsofunittesting,withaminimumamountoftimeandeffort.

UNITTESTINGBESTPRACTICES Let’slookatsomebestpracticesforbuilding,running,andmaintainingunittests,toachievethebestresults.

UnitTestsShouldBeTrustworthy Thetestmustfailifthecodeisbrokenandonlyifthecodeisbroken.Ifitdoesn't,wecannottrustwhatthetestresultsaretellingus.

UnitTestsShouldBeMaintainableandReadable Whenproductioncodechanges,testsoftenneedtobeupdated,andpossiblydebuggedaswell.So,itmustbeeasytoreadandunderstandthetest,notonlyforwhoeverwroteit,butforotherdevelopersaswell.Alwaysorganizeandnameyourtestsforclarityandreadability.

UnitTestsShouldVerifyaSingleUseCase Goodtestsvalidateonethingandonethingonly,whichmeansthattypically,theyvalidateasingleuse-case.Teststhatfollowthisbestpracticearesimplerandmoreunderstandable,andthatisgoodformaintainabilityanddebugging.Teststhatvalidatemorethanonethingcaneasilybecomecomplexandtime-consumingtomaintain.Don'tletthishappen.

Anotherbestpracticeistouseaminimalnumberofassertions.Somepeoplerecommendjustoneassertionpertest(thismaybealittletoorestrictive);theideaistofocusonvalidatingonlywhatisneededfortheuse-caseyouaretesting.

UnitTestsShouldBeIsolated Testsshouldberunnableonanymachine,inanyorder,withoutaffectingeachother.Ifpossible,testsshouldhavenodependenciesonenvironmentalfactorsorglobal/externalstate.Teststhathavethesedependenciesarehardertorunandusuallyunstable,makingthemhardertodebugandfix,andendupcostingmoretimethantheysave(seetrustworthy,above).

MartinFowler,afewyearsago,wroteabout"solitary"vs"sociable"code,todescribedependencyusageinapplicationcode,andhowtestsneedtobedesignedaccordingly.Inhisarticle,"solitary"codedoesn'tdependonotherunits(it'smoreself-contained),whereas"sociable"codedoesinteractwithothercomponents.Iftheapplicationcodeissolitary,thenthetestissimple,butforsociablecodeundertest,youcaneitherbuilda"solitary"or"sociable"test.A"sociabletest"wouldrelyonrealdependenciesinordertovalidatebehavior,whereasa"Solitarytest"isolatesthecodeundertestfromdependencies.ThisisvisuallyshowninFigure1.Youcanusemockstoisolatethecodeundertestandbuilda"solitary"testfor"sociable"code.We'lllookathowtodothatbelow.

Ingeneral,usingmocksfordependenciesmakesourlifeeasierastesters,becausewecangenerate"solitarytests"forsociablecode.Asociabletestforcomplexcodemayrequirealotofsetupandmayviolatetheprinciplesofbeingisolatedandrepeatable.Butsincethemockiscreatedandconfiguredinthetest,itisself-contained,andwehavemorecontroloverthebehaviorofdependencies.Plus,wecantestmorecodepaths.Forinstance,youcanreturncustomvaluesorthrowexceptionsfromthemock,inordertocoverboundaryorerrorconditions.

UnitTestsShouldBeAutomated Makesuretestsarebeingruninanautomatedprocess.Thiscanbedaily,oreveryhour,orinaContinuousIntegrationorDeliveryprocess.Thereportsneedtobeaccessibletoandreviewedbyeveryoneontheteam.Asateam,talkaboutwhichmetricsyoucareabout:codecoverage,modifiedcodecoverage,numberoftestsbeingrun,performance,etc.Alotcanbelearnedbylookingatthesenumbers,andabigshiftinthosenumbersoftenindicatesregressionsthatcanbeaddressedimmediately.

UseaGoodMixtureofUnitandIntegrationTests MichaelCohn'sbook,SucceedingwithAgile:SoftwareDevelopmentUsingScrum,addressesthisusingatestingpyramidmodel(seeillustrationinFigure2).Thisisacommonlyusedmodeltodescribetheidealdistributionoftestingresources.Theideaisthatasyougoupthepyramid,testsareusuallymorecomplextobuild,morefragile,slowertorun,andslowertodebug.Lowerlevelsaremoreisolatedandmoreintegrated,faster,andsimplertobuildanddebug.Therefore,automatedunittestsshouldmakeupthebulkofyourtests.

Figure 1: Comparison of Sociable and Solitary Tests. Source: Martin Fowler, 2014, "UnitTest"

Figure 2: Testing Pyramid Model

Unittestsshouldvalidateallofthedetails,thecornercasesandboundaryconditions,etc.Component,integration,UI,andfunctionaltestsshouldbeusedmoresparingly,tovalidatethebehavioroftheAPIsorapplicationasawhole.Manualtestsshouldbeaminimalpercentageoftheoverallpyramidstructurebutarestillusefulforreleaseacceptanceandexploratorytesting.Thismodelprovidesorganizationswithahighlevelofautomationandtestcoverage,sothattheycanscaleuptheirtestingeffortsandkeepthecostsassociatedwithbuilding,running,andmaintainingtestsataminimum.

UnitTestsShouldBeExecutedWithinanOrganizedTestPractice Inordertodrivethesuccessofyourtestingatalllevels,andmaketheunit testingprocessscalableandsustainable,youwillneedsomeadditionalpractices inplace.Firstofall,thismeanswritingunittestsasyouwriteyourapplicationcode.Someorganizationswritethetestsbeforetheapplicationcode(test-driven or behavior-drivenprogramming).Theimportantthingisthattestsgohand-in-handwiththeapplicationcode.Thetestsandapplicationcodeshouldevenbereviewedtogetherinthecodereviewprocess.Reviewshelpyouunderstandthecodebeingwritten(becausetheycanseetheexpectedbehavior)andimproveteststoo!

Writingtestsalongwithcodeisn'tjustfornewbehaviororplannedchanges,it’scriticalforbugfixestoo.Everybugyoufixshouldhaveatestthatverifiesthebug isfixed.Thisensuresthatthebugstaysfixedinthefuture.

Adoptazero-tolerancepolicyforfailingtests.Ifyourteamisignoringtestresults, thenwhyhavetestsatall?Testfailuresshouldindicaterealissuessoaddress thoseissuesrightaway—beforetheywasteQA'stime,orworse,theygetintothereleasedproduct.

Thelongerittakestoaddressfailures,themoretimeandmoneythosefailureswillultimatelycostyourorganization.So,youshouldruntestsduringrefactoring,runtestsrightbeforeyoucommitcode,anddon'tletataskbeconsidered"done"untilthetestsarepassingtoo.

Finally,maintainthosetests.Asmentionedpreviously,ifyou'renotkeepingthosetestsuptodatewhentheapplicationchanges,theylosetheirvalue.Especiallyiftheyarefailing,failingtestsarecostingtimeandmoneytoinvestigateeachtimetheyfail.Refactorthetestsasneeded,whenthecodechanges.

Asyoucansee,maximizingyourreturnsonmoneyandtimeinvestedinyourunittestsrequiressomeinvestmentinapplyingbestpractices.Butintheend,therewardsareworththeinitialinvestment.

WHATABOUTCODECOVERAGE? Ingeneral,codecoverageisameasurementofhowmuchoftheproductioncodeisexecutedwhileyourautomatedtestsarerunning.Byrunningasuiteoftestsandlookingatcodecoveragedata,youcangetageneralsenseofhowmuchofyourapplicationisbeingtested.

Therearemanykindsofcodecoverage—themostcommononesarelinecoverageandbranchcoverage.Mosttoolsfocusonlinecoverage,whichjusttellsyouifaspecificlinewascovered.Branchismoregranular,asittellsyouifeachpaththroughthecodeiscovered.

Codecoverageisanimportantmetricbutrememberthatincreasingitisameanstoanend.It’sgreatforfindinggapsintesting,butit'snottheonlythingtofocuson.Becarefulnottospendtoomuchefforttryingtoachieve100%coverage–itmaynotevenbepossibleorfeasible,andreallythequalityofyourtestsistheimportantthing.Thatbeingsaid,achievingatleast60%coverageforyourprojectsisagoodstartingpoint,and80%ormoreisagoodgoaltoset.Obviously,it'suptoyoutodecidewhatthatgoalshouldbe.

It'salsovaluableifyouhaveautomatedtoolsthatnotonlymeasurecodecoveragebutalsokeeptrackhowmuchmodifiedcodeisbeingcoveredbytests,becausethiscanprovidevisibilityintowhetherenoughtestsarebeingwrittenalongwithchangesinproductioncode.

Figure3showsanexamplecodecoveragereportfromParasoftDTP,thereporting andanalyticshub,thatyoucannavigatethroughifyouareusingParasoftJtestforyourunittesting:

Figure 3: Example Code Coverage Report

Anotherthingtokeepinmindisthat,whenwritingnewtests,becarefuloffocusingonlinecoveragealone,assinglelinesofcodecanresultinmultiplecodepaths,somakesureyourtestsvalidatethesecodepaths.Linecoverageisausefulquickindicator,butitisn'ttheonlythingtolookfor.

Themostobviouswaytoincreasecoverageissimplytoaddmoretestsformore codepaths,andmoreusecasesofthemethodundertest.Apowerfulwaytoincreasecoverageistouseparameterizedtests.ForJUnit4,therewasthebuiltinJUnit4Parameterizedfunctionalityandthird-partylibrarieslikeJunitParams.JUnit5has built-inparameterization.

Finally,ifyouaren'talreadytrackingtestcoverage,wehighlyrecommendyoustart.Thereareplentyoftoolsthatcanhelp,likeParasoftJtest.Startbymeasuringyourcurrentcoveragenumbers,thensetgoalsforwhereitshouldbe,addressimportantgapsfirst,andthenworkfromthere.

SUMMARYOFUNITTESTINGBESTPRACTICES Althoughunittestingisaproventechniqueforensuringsoftwarequality,it’sstillconsideredaburdentodevelopersandmanyteamsarestillstrugglingwithit.Inordertogetthemostoutoftestingandautomatedtestingtools,testsmustbetrustworthy,maintainable,readable,self-contained,andbeusedtoverifyasingleusecase.Automationiskeytomakingunittestingworkableandscalable.

Inaddition,softwareteamsneedtopracticegoodtestingtechniques,suchaswritingandreviewingtestsalongsideapplicationcode,maintainingtests,andensuringthatfailedtestsaretrackedandremediatedimmediately.Adoptingtheseunittestingbestpracticescanquicklyimproveyourunittestingoutcomes.

JUNIT TUTORIAL: SETTING UP, WRITING, AND RUNNING JAVA UNIT TESTSThistutorialwillhelpyouunderstandthebasicsandscaleyourunittestingpracticelikeapro.

BeforewegointoJUnits,let'stalkalittlebitaboutunittestingandregressiontestingandwhytheymatteringeneral.We'llgetintosomegoodexamplesaswegoon.

UNITTESTING Unittestingisaformofwhiteboxtesting,inwhichtestcasesarebasedoninternalstructure.Thetesterchoosesinputstoexploreparticularpathsanddeterminestheappropriateoutput.Thepurposeofunittestingistoexaminetheindividualcomponentsorpieceofmethods/classestoverifyfunctionality,ensuringthebehaviorisasexpected.

Theexactscopeofa“unit”isoftenlefttointerpretation,butaniceruleofthumbisforaunittocontaintheleastamountofcodethatperformsastandalonetask(e.g.asinglemethodorclass).Thereisagoodreasonthatwelimitscopewhenunittesting--ifweconstructatestthatincorporatesmultipleaspectsofaproject,wehaveshiftedfocus,fromfunctionalityofasinglemethod,tointeractionbetweendifferentportionsofthecode.Ifthetestfails,wedon'tknowwhyitfailed,andweareleftwonderingwhetherthepointoffailurewaswithinthemethodwewereinterestedin,orinthedependenciesassociatedwiththatmethod.

REGRESSIONTESTING Complementingunittesting,regressiontestingmakescertainthatthelatestfix,enhancement,orpatchdidnotbreakexistingfunctionality,bytestingthechangesyou'vemadetoyourcode.Changestocodeareinevitable,whethertheyaremodificationsofexistingcodeoraddingpackagesfornewfunctionality--yourcodewillcertainlychange.Itisinthischangethatthemostdangerlies,sowiththatinmind,regressiontestingisamust.

WHATISJUNIT? JUnitisaunittestingframeworkfortheJavaprogramminglanguagethatplaysabigroleinregressiontesting.Anopen-sourceframework,itisusedtowriteandrunrepeatableautomatedtests.

Aswithanythingelse,theJUnitframeworkhasevolvedovertime.ThemajorchangetomakenoteofistheintroductionofannotationsthatcamealongwiththereleaseofJUnit4,whichprovidedanincreaseinorganizationandreadabilityofJUnits.TherestofthisblogpostwillbewrittenfromusagesofJUnit4and5.

HOWTOSETUPJUNIT ThemorecommonIDEs,suchasEclipseandIntelliJ,willalreadyhaveJUnitfunctionalityinstalledbydefault.IfoneisnotusinganIDEandperhapsrelyingsolelyonabuildsystemsuchasMavenorGradle,theinstallationofJUnit4/5ishandledviathepom.xmlorbuild.gradle,respectively.ItisimportanttonotethatJUnit5wassplitintothreemodules,oneofthosebeingavintagemodulethatsupportsannotation/syntaxofJUnit4.

JUNIT4 ToaddJUnit4toyourMaven,buildthefollowingtothepom.xml. Bemindfulofversion:

<dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.12</version> <scope>test</scope> </dependency>

ForGradle,addthefollowingtothebuild.gradle:

apply plugin: 'java'

dependencies { testCompile 'junit:junit:4.12' }

JUNIT5 AddingJUnit5isabitdifferent.BecauseofthemodularfashionofJUnit5,aBOMisusedtoimportallaspects.Ifonlyparticularclassesareneeded,individualgroupsorartifactscanbespecified.

ToaddJUnit5toMaven,addthefollowingtopom.xml:

<dependency> <groupId>org.junit</groupId> <artifactId>junit-bom</artifactId> <version>5.2.0</version> <scope>test</scope> </dependency>

ForGradle,addthefollowingtothebuild.gradle:

apply plugin: 'java'

dependencies { implementation 'org.junit:junit-bom:5.2.0' }

Althoughnottypicallyneeded,theraw.jarfile,whichallowsonetousetheJUnitframework,isalsoaccessibletomanuallyputontheclasspath.GithousesthecodeforJUnit.JUnit4hasthe.jaravailabletodownloaddirectly.ItismostcommontoincludetheJUnit5.jarfilesusingabuildmanagementsystem,likeMavenorGradle.SeetheJUnit5docsonlineforspecifics.

WRITINGUNITTESTS:THEANATOMYOFAJUNIT Nowthatwetalkedalittleaboutunittestingandsetup,let'smoveontoactualconstructionandexecutionofthesetests.TobestillustratethecreationofJUnits, wewanttostartwithsomethingbasic.Intheexampleimagebelow,wehavea simplemethod(left)thatconvertsFahrenheittoCelsius,andtheJUnit(right)associatedwithourmethod.TheJUnitsarenumberedinsectionsbelowandwe'lldiscusseachindetail.

Sections1and2 TheseareimportsfortheJUnitlibrariesneededtoleveragethetestingframework.TheimportedlibrariescanbespecifieddowntoaparticularfunctionalityofJUnitbutarecommonlyimportedwithasteriskstohaveaccesstoallfunctionality.

Section3 Thishasthestartofourtestclass,andtheimportantthingtotakenoteofhereisthenamingconventionfortheclass,whichfollowsClassNameTest.

Section4 Here,weseeourfirstJUnit-specificsyntax,anannotation.AnnotationsareextremelyimportantwhencreatingJUnits.ThisishowJUnitknowswhattodowiththeprocessingsectionofcode.Inourexamplecase,wehavean@Testannotation,whichtellsJUnitthatthepublicvoidmethodtowhichitisattachedcanberunasatestcase.

Figure 4: Example Code With Example Unit Test

Therearemanyotherannotations,butthemorecommonare@Before(whichrunssomestatement/preconditionbefore@Test,publicvoid),@After(whichrunssomestatementafter@Test,publicvoide.g.resettingvariables,deletingtemporaryfiles,variables,etc.),and@Ignore(whichignoressomestatementduringtestexecution--notethat@BeforeClassand@AfterClassareusedforrunningstatementsbeforeandafteralltestcases,publicstaticvoid,respectively).

Section5 Thetakeawayhereisagainnamingconvention.Notethestructure,testMethodName.

Section6 Hereweconstructanewinstanceofourclassobject.Thisisnecessarysowecancallthemethodwearetestingonsomething.Withoutthisobjectinstance,wecannottestthemethod.

Section7 Variablesassociatedwiththemethodneedtobeestablished,soherewedeclarevariablescorrespondingtoourmethod.Theseshouldbegivenmeaningfulvalues(note:ifaparameterisanobject,onecaninstantiateit,ormockit),sothatourtest hasmeaning.

Section8 Thisvariabledeclarationcouldbearguedasoptional,butit'sworthwhileforthesakeoforganizationandreadability.Weassigntheresultsofourmethodbeingtestedtothisvariable,usingitasneededforassertionsandsuch.

Section9 Theassertmethods(whicharepartoftheorg.junit.Assertclass)areusedindeterminingpass/failstatusoftestcases.Onlyfailedassertionsarerecorded.Likewithannotations,therearemanyassertoptions.InourexampleJUnitabove,weuseassertEquals(expected,actual,delta).Thistakesintheexpectedoutcome,whichtheuserdefines,theactual,whichistheresultofthemethodbeingcalled,andthedelta,whichallowsforimplementinganalloweddeviationbetweenexpectedandactualvalues.Thepurposeofanassertionisvalidation.AlthoughnotrequiredtorunyourJUnit,failingtoaddassertionsarguablydefeatsthepurposeofyourtest.Withoutassertions,youhavenoverificationandatmostasmoketest,whichgivesfeedbackonlywhenatesterrorsout.

HOWTORUNAJUNIT Chooseyourownadventure!Here,wewilllookatthreewaystorunJUnits:straightfromthecommandline,fromtheIDE(EclipseandIntelliJ),andusingbuildsystems(MavenandGradle).

How to Run a JUnit From the Command Line TorunaJUnitdirectlyfromthecommandline,youneedafewthings:JDKonyourpath,rawJunitjarfile,andthetestcases.Thecommandisasfollows(thisexampleisforJUnit4):

java -cp /path/to/junit.jar org.junit.runner.JUnitCore <test class name>

NOTE: it is unlikely in a professional setting that one would be running a test manually from the command line, without some build system, but the ability is there.

How to Run a JUnit From the IDE Eclipse TorunfromEclipse,fromyourPackageExplorerlocateyourJUnittest,inwhicheverfolderyouhavedesignateditto.Right-click,andmovedowntoRunAsJUnitTest.ThiswillexecuteyourtestandopenanewJUnitwindowifnotalreadyopen.

Figure 5: Eclipse Menu to Run as JUnit Test

IntelliJ RunningatestinIntelliJisverysimilartoEclipse.FromtheProjectwindow,locatetest,right-click,andselectRun‘testName’.LikeEclipse,aJUnitwindowwillopenwiththeresultsofthetest.

Figure 6: IntelliJ Menu to Run a Unit Test

How to Run a JUnit Using Build Systems

Maven Mavenmaderunningtestssimple.Ensureyouareintheproperlocationfromyourcommandline,andtheprojectpom.xmlisproperlyconfigured.ThenyoucanrunthefollowingtoexecuteyourJUnits:

Torunentiretestsuite:

mvn test

Torunsingle/specifictest(s):

mvn -Dtest=TestName test

Gradle Gradle,likeMaven,maderunningtestssimple.

Torunentiretestsuite:

gradlew test

Torunsingle/specifictest(s):

gradlew -Dtest.single=testName test

NOTE: Maven and Gradle are their own monster -- what is shown here is minimal to cover the basics.

CONTINUINGWITHUNITTESTING Ourexampleranthroughaverysimplesnippetofcode,andofcourse,thisisjustthestartofunittesting.Morecomplexmethodscalldatabasesorothermethods,buttoreassurefunctionalityweneedisolation,whichweachievethroughmocking.Mockinghelpsusisolateunitsofcodetofocusourvalidation.FrameworkscommonlyusedformockingareMockitoandPowerMock.

Thebenefitsofunittestingareclear:

» Identifydefectswithisolationandfocusedtesting.

» Assurebehaviorofindividualmethodsorpiecesofmethods.

» Helpstoensureadditionormodificationofcodedoesnotbreaktheapplication.

» Boundaryanalysismakesiteasiertocheckforinvalid/badinput.

» Testeveryaspectofthemethodtoincreasecodecoverage.

It'shelpfultodeploypowerfulunittestingtoolslikeParasoftJtestthatcanremedymuchofthepainassociatedwithJUnitsandsavedevelopersvaluabletime.

MOCKING IN JAVA: HOW TO AUTOMATE A JAVA UNIT TEST, INCLUDING MOCKING AND ASSERTIONSWhatismockinginJava?Youcanauto-generateaunittestwithasinglebuttonclick,includingallofthemockingandvalidations.

Goodunittestsareagreatwaytomakesurethatyourcodeworkstodayandcontinuestoworkinthefuture.Acomprehensivesuiteoftests,withgoodcode-basedandbehavior-basedcoverage,cansaveanorganizationalotoftimeandheadaches.Andyet,itisnotuncommontoseeprojectswherenotenoughtestsarewritten.Infact,somedevelopershaveevenbeenarguingagainsttheirusecompletely.

WHATMAKESAGOODUNITTEST? Therearemanyreasonswhydevelopersdon’twriteenoughunittests.Oneofthebiggestreasonsistheamountoftimetheytaketobuildandmaintain,especiallyinlarge,complexprojects.Incomplexprojects,oftenaunittestneedstoinstantiateandconfigurealotofobjects.Thistakesalotoftimetosetupandcanmakethetestascomplex(ormorecomplex)thanthecodeitistesting,itself.

Let’slookatanexampleinJava:

public LoanResponse requestLoan(LoanRequest loanRequest, LoanStrategy strategy) { LoanResponse response = new LoanResponse(); response.setApproved(true); if (loanRequest.getDownPayment().compareTo(loanRequest.getAvailableFunds()) > 0) { response.setApproved(false); response.setMessage("error.insufficient.funds.for.down.payment"); return response; } if (strategy.getQualifier(loanRequest) < strategy.getThreshold(adminManager)) { response.setApproved(false); response.setMessage(getErrorMessage()); } return response; }

HerewehaveamethodthatprocessesaLoanRequest,generatingaLoanResponse.NotetheLoanStrategyargument,whichisusedtoprocesstheLoanRequest.Thestrategyobjectmaybecomplex–itmayaccessadatabase,anexternalsystem,orthrowaRuntimeException.TowriteatestforrequestLoan(),weneedtobeawareofwhichtypeofLoanStrategywe'retestingwithandweprobablyneedtotestthemethodwithavarietyofLoanStrategyimplementationsandLoanRequestconfigurations.

AunittestforrequestLoan()maylooklikethis:

Asyoucansee,there’sawholesectionofourtestwhichjustcreatesobjectsandconfiguresparameters.Itwasn’tobviouslookingattherequestLoan()methodwhatobjectsandparametersneedtobesetup.Tocreatethisexample,wehadtorunthetest,addsomeconfiguration,thenre-runagainandrepeattheprocessoverandover.WespenttoomuchtimefiguringouthowtoconfiguretheAdminManagerandtheLoanStrategyinsteadoffocusingonourmethodandwhatneededtobetestedthere.AndIstillneedtoexpandourtesttocovermoreLoanRequestcases,morestrategies,andmoreparametersforAdminDao.

Additionally,byusingrealobjectstotestwith,thistestisactuallyvalidatingmorethanjustthebehaviorofrequestLoan()—we'redependingonthebehaviorofAvailableFundsLoanStrategy,AdminManagerImpl,andAdminDaoinorderforourtesttorun.Effectively,we'retestingthoseclassestoo.Insomecases,thisisdesirable,butinothercasesitisnot.Plus,ifoneofthoseotherclasseschanges,thetestmaystartfailingeventhoughthebehaviorofrequestLoan()didn’tchange.Forthistest,wewouldratherisolatetheclassundertestfromitsdependencies.

@Test public void testRequestLoan() throws Throwable { // Set up objects DownPaymentLoanProcessor processor = new DownPaymentLoanProcessor();

LoanRequest loanRequest = LoanRequestFactory.create(1000, 100, 10000); LoanStrategy strategy = new AvailableFundsLoanStrategy();

AdminManager adminManager = new AdminManagerImpl(); underTest.setAdminManager(adminManager); Map<String, String> parameters = new HashMap<>(); parameters.put("loanProcessorThreshold", "20");

AdminDao adminDao = new InMemoryAdminDao(parameters); adminManager.setAdminDao(adminDao);

// Call the method under test LoanResponse response = processor.requestLoan(loanRequest, strategy);

// Assertions and other validations }

MOCKINGINJAVA Onesolutionforthecomplexityproblemistomockthosecomplexobjects.Forthisexample,let'sstartbyusingamockfortheLoanStrategyparameter:

@Test public void testRequestLoan() throws Throwable { // Set up objects DownPaymentLoanProcessor processor = new DownPaymentLoanProcessor(); LoanRequest loanRequest = LoanRequestFactory.create(1000, 100, 10000); LoanStrategy strategy = Mockito.mock(LoanStrategy.class);

Mockito.when(strategy.getQualifier(any(LoanRequest.class))).thenReturn(20.0d);

Mockito.when(strategy.getThreshold(any(AdminManager.class))).thenReturn(20.0d);

// Call the method under test LoanResponse response = processor.requestLoan(loanRequest, strategy);

// Assertions and other validations }

Let’slookatwhat’shappeninghere.WecreateamockedinstanceofLoanStrategy usingMockito.mock().SinceweknowthatgetQualifier()andgetThreshold() willbecalledonthestrategy,wedefinethereturnvaluesforthosecallsusingMockito.when(…).thenReturn().Forthisthetest,wedon’tcarewhattheLoanRequestinstance’svaluesare,nordoweneedarealAdminManageranymorebecauseAdminManagerwasonlyusedbytherealLoanStrategy.

Additionally,sincewearen'tusingarealLoanStrategy,wedon’tcarewhattheconcreteimplementationsofLoanStrategymightdo.Wedon’tneedtosetuptestenvironments,dependencies,orcomplexobjects.WearefocusedontestingrequestLoan()–notLoanStrategyorAdminManager.Thecode-flowofthemethodundertestisdirectlycontrolledbythemock.

ThistestisaloteasiertowritewithMockitothanhavingtocreateacomplexLoanStrategyinstance.Buttherearestillsomechallenges.

» Forcomplexapplications,testsmayrequirelotsofmocks.

» IfyouarenewtoMockito,youneedtolearnitssyntaxandpatterns.

» Youmaynotknowwhichmethodsneedtobemocked.

» Whentheapplicationchanges,thetests(andmocks)needtobeupdatedtoo.

SOLVINGMOCKINGCHALLENGESWITHAJAVAUNITTESTGENERATOR WedesignedParasoftJtesttohelpaddressthechallengesaboveandmanagetherisksofJavasoftwaredevelopment.

Ontheunittestingsideofthings,ParasoftJtesthelpsyouautomatesomeofthemostdifficultpartsofcreatingandmaintainingunittestswithmocks.Fortheaboveexample,itcanauto-generateatestforrequestLoan()withasinglebutton-click,includingallofthemockingandvalidationsyouseeintheexampletest.

Below,the“Regular”actionintheParasoftJtestUnit Test AssistantToolbargeneratesthefollowingtest:

@Test public void testRequestLoan() throws Throwable { // Given DownPaymentLoanProcessor underTest = new DownPaymentLoanProcessor(); // When double availableFunds = 0.0d; // UTA: default value double downPayment = 0.0d; // UTA: default value double loanAmount = 0.0d; // UTA: default value

LoanRequest loanRequest = LoanRequestFactory.create(availableFunds, downPayment, loanAmount); LoanStrategy strategy = mockLoanStrategy(); LoanResponse result = underTest.requestLoan(loanRequest, strategy); // Then // assertNotNull(result); }

Allthemockingforthistesthappensinahelpermethod:

private static LoanStrategy mockLoanStrategy() throws Throwable { LoanStrategy strategy = mock(LoanStrategy.class); double getQualifierResult = 0.0d; // UTA: default value when(strategy.getQualifier(any(LoanRequest.class))).thenReturn(getQualifierResult);

double getThresholdResult = 0.0d; // UTA: default value

Figure 7: Test Generation With Parasoft Jtest

when(strategy.getThreshold(any(AdminManager.class))).thenReturn(getThresholdResult);

return strategy; }

Allthenecessarymockingissetupforus—ParasoftJtestdetectedthemethodcallstogetQualifier()andgetThreshold()andmockedthemethods.OnceweconfigurevaluesintheunittestforavailableFunds, downPayment,etc,thetestisreadytorun(wecouldalsogenerateaparameterizedtestforbettercoverage!).Notealsothattheassistantprovidessomeguidanceastowhichvaluestochangebyitscomments,“UTA:defaultvalue”,makingtestingeasier.

Thissavesalotoftimeingeneratingtests,especiallyifwedon’tknowwhatneedstobemockedorhowtousetheMockitoAPI.

HANDLINGCODECHANGES Whentheapplicationlogicchanges,thetestsoftenneedtochangealso.Ifthetestiswell-written,itshouldfailifyouupdatethecodewithoutupdatingthetest.Often,thebiggestchallengeinupdatingthetestisunderstandingwhatneedstobeupdated,andhowexactlytoperformthatupdate.Iftherearelotsofmocksandvalues,itcanbedifficulttotrackdownwhatthenecessarychangesare.

Toillustratethis,let’smakesomechangestothecodeundertest:

public LoanResponse requestLoan(LoanRequest loanRequest, LoanStrategy strategy) { ... String result = strategy.validate(loanRequest); if (result != null && !result.isEmpty()) { response.setApproved(false); response.setMessage(result); return response; } ... return response; }

WehaveaddedanewmethodtoLoanStrategy – validate(),andarenowcallingitfromrequestLoan().Thetestmayneedtobeupdatedtospecifywhatvalidate()shouldreturn.

Withoutchangingthegeneratedtest,let’srunitwithintheParasoftJtestUnit TestAssistant:

ParasoftJtestdetectedthatvalidate()wascalledonthemockedLoanStrategy argumentduringourtestrun.Sincethemethodhasnotbeensetupforthemock,theassistantrecommendsthatwemockthevalidate()method.The“Mockit”quick-fixactionupdatesthetestautomatically.Thisisasimpleexample–butforcomplexcodewhereitisn’teasytofindthemissingmock,therecommendationandquick-fixcansaveusalotofdebuggingtime.

Afterupdatingthetestusingthequickfix,wecanseethenewmockandsetthedesiredvalueforvalidateResult:

private static LoanStrategy mockLoanStrategy() throws Throwable {

LoanStrategy strategy = mock(LoanStrategy.class); String validateResult = ""; // UTA: default value

when(strategy.validate(any(LoanRequest.class))).thenReturn(validateResult); double getQualifierResult = 20.0d;

when(strategy.getQualifier(any(LoanRequest.class))).thenReturn(getQualifierResult); double getThresholdResult = 20.0d;

when(strategy.getThreshold(any(AdminManager.class))).thenReturn(getThresholdResult); return strategy; }

WecanconfigurevalidateResultwithanon-emptyvaluetotesttheusecasewherethemethodentersthenewblockofcode,orwecanuseanemptyvalue(ornull)tovalidatebehaviorwhenthenewblockisnotentered.

Figure 8: Example of Mocking in Parasoft Jtest

ANALYZINGTHETESTFLOW Theassistantalsoprovidessomeusefultoolsforanalyzingthetestflow.Forinstance,hereistheflowtreeforourtestrun:

SUMMARYOFMOCKINGINJAVA Youcanautomatemanyaspectsofunittesting.ParasoftJtesthelpsyougenerateunittestswithlesstimeandeffort,reducingthecomplexityassociatedwithmocking.Italsomakesmanyotherrecommendationstoimproveexistingtestsbasedonruntimedata,andhassupportforparameterizedtests,SpringApplicationtests,andPowerMock(formockingstaticmethodsandconstructors).

Figure 9: The Parasoft Jtest Unit Test Assistant’s Flow Tree, showing calls made during test execution.

HOW TO CREATE JUNIT PARAMETERIZED TESTSParameterizedtestsareagoodwaytodefineandrunmultipletestcases,wheretheonlydifferencebetweenthemisthedata.Here,welookatthreedifferentframeworkscommonlyusedwithJUnittests.

Whenwritingunittests,itiscommontoinitializemethodinputparametersandexpectedresultsinthetestmethoditself.Insomecases,usingasmallsetofinputsisenough;however,therearecasesinwhichweneedtousealargesetofvaluestoverifyallofthefunctionalityinourcode.Parameterizedtestsareagoodwaytodefineandrunmultipletestcases,wheretheonlydifferencebetweenthemisthedata.Theycanvalidatecodebehaviorforavarietyofvalues,includingbordercases.Parameterizingtestscanincreasecodecoverageandprovideconfidencethatthecodeisworkingasexpected.

ThereareanumberofgoodparameterizationframeworksforJava.Inthisarticle,wewilllookatthreedifferentframeworkscommonlyusedwithJUnittests,withacomparisonbetweenthemandexamplesofhowthetestsarestructuredforeach.Finally,wewillexplorehowtosimplifyandexpeditethecreationofparameterizedtests.

JUNITPARAMETERIZEDTESTFRAMEWORKS Let’scomparethe3mostcommonframeworks:JUnit4,JunitParams,andJUnit5.EachJUnitparameterizationframeworkhasitsownstrengthsandweaknesses.

JUnit4 Pros » ThisistheparameterizationframeworkbuiltintoJUnit4,soitrequiresnoadditional externaldependencies. » ItsupportsolderversionsofJava(JDK7andolder).

Cons » Testclassesusefieldsandconstructorstodefineparameters,whichmaketests moreverbose. » Itrequiresaseparatetestclassforeachmethodbeingtested.

JunitParams Pros » Simplifiesparametersyntaxbyallowingparameterstobepasseddirectly toatestmethod. » Allowsmultipletestmethods(eachwiththeirowndata)pertestclass. » SupportsCSVdatasources,aswellasannotation-basedvalues (nomethodrequired).

Cons » RequirestheprojecttobeconfiguredwiththeJunitParamsdependency. » Whenrunninganddebuggingtests,alltestswithintheclassmustberun—it isnotpossibletorunasingletestmethodwithinatestclass.

JUnit5 Pros » ThisparameterizationframeworkisbuiltintoJUnit5andimproveswhatwas includedwithJUnit4. » HasasimplifiedparametersyntaxlikeJunitParams. » Supportsmultipledata-setsourcetypes,includingCSVandannotation (nomethodrequired). » Eventhoughnoextradependenciesarerequired,morethanone.jarisneeded.

Consideration » RequiresnewerversionsofJavaandyourpreferredbuildsystem(e.g.Gradle orMavenSurefire).ChecktheJUnit5specificationsfordetails.

EXAMPLEOFAJUNITPARAMETERIZEDTEST Asanexample,supposethatwehaveamethodthatprocessesloanrequestsforabank.Wemightwriteaunittestthatspecifiestheloanrequestamount,downpaymentamount,andothervalues.Wewouldthencreateassertionsthatvalidatetheresponse—theloanmaybeapprovedorrejected,andtheresponsemayspecifythetermsoftheloan.

public LoanResponse requestLoan(float loanAmount, float downPayment, float availableFunds)

{ LoanResponse response = new LoanResponse(); response.setApproved(true);

if (availableFunds < downPayment) { response.setApproved(false); response.setMessage("error.insufficient.funds.for.down.payment");

return response; }

if (downPayment / loanAmount < 0.1) { response.setApproved(false); response.setMessage("error.insufficient.down.payment"); }

return response; }

Viewraw. Viewparameterizedtestexample.

First,let’slookataregulartestfortheabovemethod:

@Test public void testRequestLoan() throws Throwable {

// Given| LoanProcessor underTest = new LoanProcessor();

// When LoanResponse result = underTest.requestLoan(1000f, 200f, 250f);

// Then assertNotNull(result); assertTrue(result.isApproved()); assertNull(result.getMessage()); }

Viewraw. Viewparameterizedtestexample2.

Inthisexample,wearetestingourmethodbyrequestinga$1000loan,witha $200downpaymentandindicatingthattherequestorhas$250inavailablefunds. Thetestthenvalidatesthattheloanwasapprovedanddidn’tprovideamessagein theresponse.

InordertomakesurethatourrequestLoan()methodistestedthoroughly,weneedtotestwithavarietyofdownpayments,requestedloanamounts,andavailablefunds.Forinstance,let’stesta$1millionloanrequestwithzerodownpayment,whichshouldberejected.Wecouldsimplyduplicatetheexistingtestwithdifferentvalues,butsincethetestlogicwouldbethesame,itismoreefficienttoparameterizethetestinstead.

Wewillparameterizetherequestedloanamount,downpayment,andavailablefunds,aswellastheexpectedresults:whethertheloanwasapproved,andthemessagereturnedaftervalidation.Eachsetofrequestdata,alongwithitsexpectedresults, willbecomeitsowntestcase.

ANEXAMPLEOFAPARAMETERIZEDTESTUSINGJUNIT4PARAMETERIZED Let’sstartwithaJUnit4Parameterizedexample.Tocreateaparameterized test,wefirstneedtodefinethevariablesforthetest.Wealsoneedtoinclude aconstructortoinitializethem:

@RunWith(Parameterized.class) public class LoanProcessorParameterizedTest {

float loanAmount; float downPayment; float availableFunds; boolean expectApproved; String expectedMessage;

public LoanProcessorParameterizedTest(float loanAmount, float downPayment,

float availableFunds, boolean expectApproved, String expectedMessage)

{ this.loanAmount = loanAmount; this.downPayment = downPayment; this.availableFunds = availableFunds; this.expectApproved = expectApproved; this.expectedMessage = expectedMessage; }

// ...

Here,weseethatthetestusesthe@RunWithannotationtospecifythatthetestwillrunwiththeJUnit4Parameterizedrunner.Thisrunnerknowstolookforamethodwhichwillprovidethevalue-setforthetest(annotatedwith@Parameters),initializethetestproperly,andrunthetestswithmultiplerows.

Notethateachparameterisdefinedasafieldinthetestclass,andtheconstructorinitializesthesevalues(youcanalsoinjectvaluesintofieldsusingthe@Parameter annotationifyoudon’twanttocreateaconstructor).Foreachrowinthevalue-set,theParameterizedrunnerwillinstantiatethetestclassandruneachtestintheclass.

Let’saddamethodwhichprovidestheparameterstotheParameterizedrunner:

@Parameters(name = "Run {index}: loanAmount={0}, downPayment={1}, availableFunds={2}, expectApproved={3}, expectedMessage={4}")

public static Iterable<Object[]> data() throws Throwable

{ return Arrays.asList(new Object[][] {

{ 1000.0f, 200.0f, 250.0f, true, null } }); }

Thevalue-setsarebuiltasaList of Objectarraysbythedata()method,whichisannotatedwith@Parameters.Notethat@Parameterssetsthenameofthetestusingplaceholders,whichwillbereplacedwhenthetestruns.Thismakesiteasiertoseevaluesintestresults,aswewillseelater.Currently,thereisonlyonerowofdata,testingacasewheretheloanshouldbeapproved.Wecanaddmorerowstoincreasecoverageofthemethodundertest.

@Parameters(name = "Run {index}: loanAmount={0}, downPayment={1}, availableFunds={2}, expectApproved={3}, expectedMessage={4}")

public static Iterable<Object[]> data() throws Throwable

{ return Arrays.asList(new Object[][] {

{ 1000.0f, 200.0f, 250.0f, true, null },

{ 1000.0f, 50.0f, 250.0f, false, "error.insufficient.down.payment" },

{ 1000.0f, 200.0f, 150.0f, false, "error.insufficient.funds.for.down.payment" } }); }

Here,wehaveonetestcasewheretheloanwouldbeapproved,andtwocasesinwhichitshouldnotbeapprovedfordifferentreasons.Wemaywanttoaddrowsinwhichzeroornegativevaluesareused,aswellastestboundaryconditions.

Wearenowreadytocreatethetestmethod:

public void testRequestLoan() throws Throwable

{ // Given LoanProcessor underTest = new LoanProcessor();

// When LoanResponse result = underTest.requestLoan(loanAmount, downPayment, availableFunds);

// Then assertNotNull(result); assertEquals(expectApproved, result.isApproved()); assertEquals(expectedMessage, result.getMessage()); }

Viewraw. Parameterizedtestexample6.

Here,wereferencethefieldswheninvokingtherequestLoan()methodandvalidatingtheresults.

JUNITPARAMSEXAMPLE TheJunitParamslibrarysimplifiesparameterizedtestsyntaxbyallowingparameterstobepasseddirectlytothetestmethod.Theparametervaluesareprovidedbyaseparatemethodwhosenameisreferencedinthe@Parametersannotation.

@RunWith(JUnitParamsRunner.class) public class LoanProcessorParameterizedTest2 {

@Parameters(method = "testRequestLoan _ Parameters") public void testRequestLoan(float loanAmount, float downPayment, float availableFunds, boolean expectApproved, String expectedMessage) throws Throwable { ... }

@SuppressWarnings("unused") private static Object[][] testRequestLoan _ Parameters() throws Throwable {

// Parameters: loanAmount={0}, downPayment={1}, availableFunds={2}, expectApproved={3}, expectedMessage={4}

return new Object[][] {

{ 1000.0f, 200.0f, 250.0f, true, null }, { 1000.0f, 50.0f, 250.0f, false, "error.insufficient.down.payment"}, { 1000.0f, 200.0f, 150.0f, false, "error.insufficient.funds.for.down.payment" } }; } }

JunitParamshastheadditionalbenefitthatitsupportsusingCSVfilestoprovidevaluesinadditiontoprovidingthevaluesincode.Thisallowsthetesttobedecoupledfromthedataanddatavaluestobeupdatedwithoutupdatingthecode.

JUNIT5EXAMPLE JUnit5addressessomeofthelimitationsandshortcomingsofJUnit4.LikeJunitParams,JUnit5alsosimplifiesthesyntaxofparameterizedtests.Themostimportantchangesinsyntaxare:

» Thetestmethodisannotatedwith@ParameterizedTestinsteadof@Test.

» Thetestmethodacceptsparametersdirectly,insteadofusingfields andaconstructor.

» The @RunWithannotationisnolongerneeded.

DefiningthesameexampleinJUnit5wouldlooklikethis:

public class LoanProcessorParameterizedTest {

@ParameterizedTest(name="Run {index}: loanAmount={0}, downPayment={1}, availableFunds={2}, expectApproved={3}, expectedMessage={4}")

@MethodSource("testRequestLoan _ Parameters") public void testRequestLoan(float loanAmount, float downPayment, float availableFunds,

boolean expectApproved, String expectedMessage) throws Throwable

{ ... }

static Stream<Arguments> testRequestLoan _ Parameters() throws Throwable {

return Stream.of(

Arguments.of(1000.0f, 200.0f, 250.0f, true, null),

Arguments.of(1000.0f, 50.0f, 250.0f, false, "error.insufficient.down.payment"),

Arguments.of(1000.0f, 200.0f, 150.0f, false, "error.insufficient.funds.for.down.payment") ); } }

EFFICIENTLYCREATEPARAMETERIZEDTESTS Asonemightimagine,writingtheaboveparameterizedtestcanbeabitofwork.Foreachparameterizedtestframeworkthereissomeboilerplatecodethatneedstobewrittencorrectly.Itcanbehardtorememberthecorrectstructure,andittakestimetowriteout.Tomakethismucheasier,youcanuseParasoftJtesttogenerateparameterizedtests,automatically,liketheonesdescribedabove.Todothis,simplyselectthemethodyouwanttogenerateatestfor(inEclipseorIntelliJ):

Thetestisgenerated,usingdefaultvaluesandassertions.Youcanthenconfigure thetestwithrealinputvaluesandassertionsandaddmoredatarowstothe data()method.

Figure 10: Select Parameterized Test Generation in Parasoft Jtest

RUNNINGTHEPARAMETERIZEDTEST ParasoftJtestcanrunparameterizedtestsdirectlyinbothEclipseandIntelliJ.

Notethatthenameofeachtest,asshown,includesinputvaluesfromthedatasetandexpectedresultvalues.Thiscanmakedebuggingthetestmucheasierwhenitfails,sincetheinputparametersandexpectedoutputsareshownforeachcase.

YoucanalsousetheRunAllactionfromParasoftJtest:

Itanalyzesthetestflowandprovidesdetailedinformationabouttheprevioustestrun.Thisallowsyoutoseewhathappenedinthetestwithoutneedingtorerunthetestwithbreakpointsordebuggingstatements.Forinstance,youcanseeparameterizedvaluesintheVariablesview:

Figure 11: The JUnit View in Eclipse

Figure 12: The Flow Tree View in Parasoft Jtest

Figure 13: The Variables View in Parasoft Jtest

SUMMARYOFCREATINGJUNITPARAMETERIZEDTESTS Eachofthethreeframeworksthatwereviewedarefinechoicesandworkwell.IfusingJUnit4,JunitParamsispreferredoverthebuilt-inJUnit4Parameterizedframework,duetothecleanerdesignofthetestclassesandtheabilitytodefinemultipletestmethodsinthesameclass.However,ifusingJUnit5,werecommendthebuilt-inJUnit5frameworksinceitaddressestheshortcomingsinJUnit4andrequiresnoextralibraries.WealsolikeusingParasoftJtest’sunittestingcapabilitiestomakecreation,execution,anddebuggingofparameterizedtestsmoreefficient.

GET MORE OUT OF UNIT TESTING AND REDUCE MAINTENANCE EFFORTS WITH RUNTIME ANALYSIS Torealizethebenefitsofunittesting,youcanobserveaunittestduringitsexecutionviaruntimeanalysis.Runtimeanalysisduringunittestexecutioniscriticaltoimprovingtestefficiencyandeffectiveness.

Unittestingisabestpracticetotestindividualunits/componentsofasoftware,butitcanbetediousandcostlyforJavadevelopers.It'spainstakingtotesteachunitforcorrectbehaviorwithmanualassertions,andisolateeachmethodwithmocking,andunittestingitselfisopentobugsandmisunderstoodbehavior.Toimprovethissituation,youcanusearuntimeanalysistooltodetectdataandcontrolflow,externaldependencies,andtocalculatetestcodecoverage.

Withthiscollecteddatafromtheruntimeanalysis,anenterprise-gradesolutionlikeParasoftJtestcanpromptthedeveloperabouthowtoimprovethetests,byautomaticallyrecommendingassertionsforcorrectbehavior,andmethodsformockingtoimprovetestisolation.ThisintegrationbetweenautomaticunittestgenerationandruntimeanalysisreducesthemanualinterventionrequiredforunittestingforJava.

BENEFITSOFUNITTESTING Unittestingisawell-knownpractice,butitsimplementationrequiresimprovementinmanyprojects.Unittesting,donewell,improvestheagilityofagileprocess,increasesqualityandsecurity,andbringslong-termcostsavings.

Unfortunately,regardlessofthesebenefits,developerscanstillstrugglewithunittesting,despitethedesiretoachievebetterresults.Theamountoftimeandeffortneededfortestcreationandmaintenancecanbetoomuchtojustifyincreasingtestingefforts.Often,testsuitesarefragilebecauseofpoorunit/objectisolationfromdependencies.Propermockingofdependenciesbecomesthebaneofsoftwaretesters,asdoescreatingtheassertionsneededtodeterminecorrectprogramlogic.Evenparameterizingtestsforscenarioscanbetediousandtimeconsuming.

Softwaredevelopmentteamsmustaddresstheseproblemswithtestcreation,isolation,andmaintenanceiftheywanttoachievethebenefitsofthoroughunittesting.Theanswerstartswithtestautomationtools,butsimplyautomatingtheexecutionoftestsandcollectingresultsisn’tenough.Runtimeanalysis,theprocess ofobservingarunningexecutableandrecordingkeymetrics,isaninnovativeway tohelpimproveunittestingcreation,mocking,andteststability.

RUNTIMEANALYSISCANIMPROVEUNITTESTING Inmostcases,developersdon’tconsiderruntimeanalysisimportantinearlystagesofunittesting.Mosttoolsareusedforcatchingerrorsthatunittestingmissed,orsimplyincalculatingcodecoverage.Butwhilethesebenefitsareimportant,runtimeanalysiscanalsoobservetheexecutionofthefirstiterationofaunittesttomakerecommendationstoimprovethetestanddetectchangestothetestruntimeenvironmentthatinterferewithteststability.

TestframeworkssuchasJUnitcreatesparsecodethatrequiresfurtherdeveloperinput.Thisworkistedious,soitcanbeautomatedtofillinmoreofthedetailsbasedontheobservedprogramlogic.Forexample,thefollowingunittestcanbeautomaticallygeneratedbyParasoftJtest:

Figure 14: Unit Test Generation in Parasoft Jtest

Similarly,forunittestswithparameterizedinputs,shownbelow:

Sincethecreatedtestsareexecutablefromthestart,theycanbeobservedbyruntimeanalysisforbothresultsandexecutionflow.Forexample,atestmayfailduetoaraisedexception,shownbelow.

DETECTINGDEPENDENCIESANDMOCKINGWITHRUNTIMEANALYSIS Inaddition,runtimetoolsobservetheexecutionpathintodependenciesandrecommendpotentialmockstoincreasetheisolationofthetest.Althoughvisualinspectionofanobjectundertestwillrevealitsdependencies,automatingthedetectionandmockingofthesedependenciessaveslotsoftediousanderror-pronework.

Figure 15: P arameterized Test Generation in Parasoft Jtest

Figure 16: Exception Error Shown in Parasoft Jtest Unit Test Assistant

Intheexamplebelow,ParasoftJtestoffersthedeveloperachoiceofwhattomockbasedontheexecutiontraceoftheunittest:

Inthiscase,addingamockablemethodpatternaddsthemethodtoalistofmockstobehandledbyamockingframeworksuchasPowerMock.

Mockingstaticconstructorsarealsopossible,asshownbelow.

IMPROVINGTESTINGFIDELITYWITHRUNTIMEANALYSIS Withfullknowledgeoftheexecutionflow,plusparametersusedinmethodcalls,runtimeanalysiscanbeusedtoprovideusefulrecommendationstothedeveloper toimprovethetestcode.Althoughassertionsareprovided,statically,whena testiscreated,theymaynotbeenabledorcorrect.Attestexecution,failedandmissingassertionstriggerwarningswhichthenleadtorecommendationstoremedytheproblem.

Figure 17: Execution Trace of Unit Test

Figure 18: Example for Mocking Constructors

Forexample,aftercreatinganewtest,andnorecommendedassertionshavebeenuncommented,youwouldseethefollowing:

Whateverhappens,itistheconstantfeedbackaboutcorrectiveactionforassertionsthatclosestheloopontestcreationtocompleteunittesting.Additionally,astheunitundertestischanged,thesechangescanbedealtwithinthesamemanner,continuallyreducingthemanualtestmaintenancerequired.

Orifanassertionfails,forexample,thefollowingisdisplayed:

Figure 19: Example of Assertion Recommendations

Figure 20: Example of Assertion Failure

IMPROVINGTESTSTABILITYWITHRUNTIMEANALYSIS Runtimeanalysiscanalsodetectchangesinthetestenvironmentduringexecutionthatimpacttheabilitytorecreateanidenticaltestenvironmentforsubsequenttests.Teststhatpassatonetimeandfaillatercanbeacauseofgreatfrustrationandlosttimeandeffort.Someexamplesofinstabilitiesthatyoucandetectwithruntimeanalysisincludethefollowing:

» Achangedsystempropertyduringatestthathasn'tchangedbacktoitsoriginalstate.Asubsequenttestmayrelyonthisproperty.

» Additionalexecutionthreadsinthebackgroundthatmayinterferewithatestrun.

» Anewfilecreationduringtestexecutionthatmightimpactsubsequentrunsiftheyrelyonthefileanditscontents.

» Modifiedstaticfieldsthatmightimpactfutureteststhatusethesesamefields.

It’scriticalthateachtestexecutionhasanidenticalstartingpoint,toensurereliableresults.Preventingtestinstabilitywithruntimedetectionremovesguessworkfromthetestdebugphase.

CONCLUSION Youcanseethatruntimeanalysisisn'tjustforcomputingcodecoverage.Runtimeanalysisduringtestexecutioniscriticaltoimprovingtestefficiencyandeffectiveness.Monitoringexecutionpathsprovidesinformationaboutdependenciestoimprovethehandlingofdependenciesandmocking.Assertionscanbemonitored,andautomaticrecommendationsimprovetestfidelity.Detectingchangesintheruntimetestenvironmentthataffectteststabilityremovesfrustrationandreducesdebuggingcyclesfortestcode.

TAKE THE NEXT STEPLearnhowParasoftJtestcanhelpyouimproveyourJava codequalityandteamproductivity.Contactustoday.

ABOUTPARASOFT Parasofthelpsorganizationscontinuouslydeliverqualitysoftwarewithitsmarket-proven,integratedsuiteofautomatedsoftwaretestingtools.Supportingtheembedded,enterprise,andIoTmarkets,Parasoft’stechnologiesreducethetime,effort,andcostofdeliveringsecure,reliable,andcompliantsoftwarebyintegratingeverythingfromdeepcodeanalysisandunittestingtowebUIandAPItesting,plusservicevirtualizationandcompletecodecoverage,intothedeliverypipeline.Bringingallthistogether,Parasoft’sawardwinningreportingandanalyticsdashboarddeliversacentralizedviewofqualityenablingorganizationstodeliverwithconfidenceandsucceedintoday’smoststrategicecosystemsanddevelopmentinitiatives—cybersecure,safety-critical,agile,DevOps,andcontinuoustesting.

IMPROVE UNIT TESTING FOR JAVA WITH AUTOMATION

Documents

Transcript of IMPROVE UNIT TESTING FOR JAVA WITH AUTOMATION

Test Automation Using Selenium WebDriver with Java · including Web, Java, Dot-net, SAP, Peoplesoft and Seibel. His previous book “Test Automation using Unified Functional Testing”

30 Ways to Improve JAVA Performance

Leveraging Aparapi to Help Improve Java Financial …...Leveraging Aparapi to Help Improve Financial Java Application Performance Shrinivas Joshi, Software Performance Engineer Abstract

Test Automation Using Selenium WebDriver with Java · Test Automation Using Selenium WebDriver with Java Navneesh Garg • Selenium WebDriver 2.0 • Learn Automation on a Web Based

jHome: Having Fun with Home Automation and … Fun with Home Automation and Java EE ... Android .NET (WS) Java ME REST JSON ... radio-frequency based;

How Voice, Scanning and Automation Improve Distribution/Fulfillment Operations

Integration and automation to improve your business processes

Marketing automation processes as a way to improve contemporary ...

Java Test Automation for REST, Web and Mobile

Course Catalog for Selenium Automation Using Java Language

Improve operational efficiency & drive growth w/ smart workflow automation

Course Catalog for Selenium Automation Using Java Language Automation Using Java...IDE. Presentation 1. Advanced Selenium Course Details 2. Installing NotePad++ 3. Installing 7-Zip

Automation instruments to measure, control, and improve ... · Automation instruments To measure, control, and improve operations. 02 The solution: Measure. Control. And profit. ...

Test Automation Using Selenium WebDriver with Javaadactin.com/store/TestAutomationusingSeleniumWebDriverJavaPreview.pdf · Test Automation Using Selenium WebDriver with Java Navneesh

Course Catalog for Selenium Automation Using Java · PDF fileCourse Catalog for Selenium Automation Using Java Language Contact Us – 510-991-7591 Page 1 of 14

Java Automation System Integration Controllerpartner.asicontrols.com/wp-content/uploads/2018/06/data_jasic_600.pdf · Java Automation System Integration Controller ... and Modbus

10 Steps Improve automation testing

QTP Open Source Test Automation Framework Keywords for JAVA

How to Improve Recruiter Efficiency with Automation

Pushing Java EE outside of the Enterprise - Home Automation