Testing Techniques For Agile Testers - Janet Gregory

1

Belgium, 2009

With Material from Lisa Crispin

Janet GregoryDragonFire Inc.

Testing Techniquesfor

Agile Testers

Click t

o buy NOW!

PDF-XChange

www.docu-track.com Clic

k to buy N

OW!PDF-XChange

www.docu-track.com

http://www.docu-track.com/buy/


Copyright 2009 Janet Gregory, DragonFire

2

Topics

Test Planning - LevelsTesting and CollaborationThe Agile Testing QuadrantsAutomationPlanning Your Test StrategyReferences

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



3

Levels of Planning – Product, Release, IterationClic

k to buy N

OW!PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



4

Release Planning ... Test Planning

Consider:Scope / FeaturesPrioritiesRisks

Budget time:For building test infrastructureFor finding test tools

What do you need to document?Consider its usefulness

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



5

Iteration Planning ... Test Planning

Consider:Doneness for each featurePriorities – which story to detail firstRisks

Budget time:For defect fixing & retesting

What do you need to document?At minimum, the acceptance test(s)

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




6

The Importance of CollaborationThe Star Experiment

Gerald Weinberg and Donald Gause,Exploring Requirements

What is the right answer?

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




7

How Many Points?Clic

k to buy N

OW!PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




8

Collaboration

Collaboration means …working together

Communication meanssharing ideas, information, decisions & solutions

Tools for collaborationFeedbackInteractionVisibility - allows people to talk about it.

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




9

Collaboration Discussion

1. Split into teams, and discuss2. Present your findings

TWO QUESTIONS….1. How does your teams communicate?2. What are the collaboration tools you use?

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



10

What are the Agile Testing Quadrants?

Method for classifying testsLooks at the purpose of the tests

not the whenTests may cross boundariesTests that are created to support the team,

often turn into regression tests

…. Let’s look at the picture!

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




11

The Agile Testing QuadrantsIntroduced by Brian Marick

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




12

The Quadrants …..Can be used as a communication tool

To the project team

To management

To explain testing in a common language

Emphasize whole-team responsibility

Focus on collaboration

Whole team participation

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




13

Use to define ‘Doneness’

Defining “doneness” for release readinessNo story is done until testedCustomer needs captured as passingtestsAutomated regression testsDelivers value“Doneness” in all quadrants

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




14

Quadrant 1Clic

k to buy N

OW!PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Benefits of TDD

Focus on internal code qualityBuilds testability into codeIncreases confidence in designProvides instant feedback to developers

Technology-facing Tests that Support the Team

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Other Benefits of Q1 Testing

Start of building quality into the productProvides refactoring supportForms the foundation of automation suite


Programmerunit tests are

the base

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




17

Q1 Toolkit

Source code managementIntegrated development environment

compile, debug, build GUI, refactorBuild tools

eg. CruiseControl, HudsonUnit test tools

xUnitMocking tools

All needed to enable collaboration


Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




18

Discussion

What can testers do to help thedevelopers?


Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Helping developers


19

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




20

Quadrant 2Clic

k to buy N

OW!PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



21

Quadrant 2 Tests

Business-facing tests that support the team

Acceptance testsExamples

Used to elicit requirementsUser experience prototypes

wire frames, mock-upsPair testing – “Show me”

Business-facing Tests that Support the Team

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Drive Development

Example (or Acceptance Test) Driven DevUse examples to demonstrate differencesExamples can become testsExpress as executable testsCode until the tests pass

Identify hidden assumptions


Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Other Benefits for Q2 Testing

Help customers achieve advance clarity

Executable tests added to regression suite

Focus is external quality

Helps define when we're done a story

Customer – developer – tester collaboration


Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Given, When, Then - BehaviouralGiven ... Preconditions

When ... Triggers, actions

Then ... Consequences, results

Simple ex. (from Gojko Adzic, Bridging the Communication Gap):

Given the customer has a balance of $1.00 and deposits$5.00 to his account

When an authorization code A-123 is received

Then the transaction is completed, and the newbalance is $6.00

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




25

Q2 Collaboration ToolkitChecklistsPersonasMind maps

brainstormingwords, ideas, tasks

Mock-ups / paper prototypesUser-centered design

Flow diagramsWhiteboards (physical and virtual)Thin slice/steel thread


Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




26

Toolkit – Turning Examples into TestsFit/FitNesse

Collaboration in software developmentTakes place of regular UITake the place of UIUse to set up dataTest different inputs faster

xUnit frameworksWorks well for developers

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




27

More Tools to Turn Examples into Tests

Behaviour-driven development toolsAnother approach to TDDFocus on examples, “should” – common language

easyB, jBehave

GUI test toolsTest UI behaviour, system test, legacy systemsSome examples:

SeleniumWatir/WatijWebTest


Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




28

Fit Example – Failing TestClic

k to buy N

OW!PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




29

Fit Example – Passing TestClic

k to buy N

OW!PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Exercise StoryAs an online shopper, I want to be able to see thecost of different shipping options when I checkout, so I can choose which one I want.

Assumptions:- User has already entered shipping address.- API for cost calculator available, takes postal code and weight.- The options are Ground, 2 day and Overnight.- PO Boxes are ground only.- Items > 20 lbs are Ground only.- User will be able to choose different options for different items.

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




31

Exercise – Let’s Question

1. What kind of shipping options are we talking about?2. What costs do we have to consider?3. Can we break this story into smaller chunks?4. Are there multiple viewpoints? Who?5. What would the first slice be?

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




32

Exercise - Let’s draw ….

The first “slice” is: …..

1. In your group, select a customer.2. Draw the process flow(s)3. Identify the steel thread or thin slice you can

start developing

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




33

Exercise – Let’s ATDD

1. Identify a high level acceptance test (s)for the first slice.

2. For the 2nd? 3rd

What did you learn?

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Story Test Planning

Think of variations

Capture assumptions, thoughts....

Story test template

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




35

Exercise – Let’s AutomateTake your acceptance testsCreate Fit type tests or Given, When, Then

Example:


Option Address PostalCode

Weight Cost

Ground Seattle 123456 5 lbs 3.52

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




36

Quadrant 3Clic

k to buy N

OW!PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Quadrant 3 TestsBusiness-facing tests that critique the product

Exploratory Testing

Test for UsabilityCollaborative TestingDemosUser Acceptance Testing

Business-facing Tests that Critique the Product

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Exploratory Testing

Learn & designSession-based (time-box)Provides feedback into new storiesTest for usability

understand end users - personaswho will be using the system

Follow “smells”, your instinctsTouring (James Bach, James Whitaker)


Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




39

Other Types of Testing

Scenario testingProcess flows

Realistic data

Soap opera testing (Hans Buwalda)

Usability testingPersonas

Navigation

Observing users

Don't forget documents, reports, help text

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Collaborative TestingProvide feedback ….

Turn learnings into tests that drive new featuresChange process as needed

Iteration reviewsBuilds confidenceQuick feedback loop

Informal demosPair exploratory testing with customerEven on unfinished code


Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Benefits for Q3 Testing

Feedback to Quadrants One and TwoEvaluation of the actual productRecreate actual user experiencesRealistic use and data

Remember your contextWhat works for your situation“It depends”; a tool, not a rule


Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




42

Tools to Critique the GUI

Record/playbackTraditional vendor toolsSome open source tools have recorders

Open source libraries for web browsersWatir is Ruby library for testing thru browser

Specifying testsCanoo WebTests specified in XML

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




43

Tools for Exploratory Testing

Test scenario setupeg. Watir/Watij scripts

Generate test dataeg. PerlClip, Ruby script, Fit

SimulatorsSimulate data, feed to app over time

MonitorsWatch log files

EmulatorsDuplicate system behaviour

eg. mobile devices

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Exploratory Test ExerciseStory: As an online shopper, I want to be able to deleteitems from my shopping cart.

Think of some personas or characters and deviseexploratory scenarios which that role might get into,for example:

Senior citizen who never shopped on the webHacker looking to cause troubleWorking mom in a rushInternet-savvy teen

44

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




45

Quadrant 4Clic

k to buy N

OW!PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Quadrant 4 TestsBusiness-facing tests that critique the product

Non-functional tests“ility” testingPerformance, scalability, stress, loadMemory managementSecurity testing

Roles & permissions , system ‘hacking’Data migrationInfrastructure TestingRecovery

Technology-facing Tests that Critique the Product

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Benefits of Q4 Testing“Non-functional” requirements may be higherpriority than “functional”

PerformanceStabilitySecurity...other quality criteria

Helps your application deliver the ‘right’ valueMakes the ‘finished’ product

Technology-facing Tests that Critique the ProductCopyright 2009 Janet Gregory, DragonFire

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Any questions about the quadrants?Clic

k to buy N

OW!PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



49

Why Automate?

Free up time for most important workRepeatableSafety netQuick feedbackHelp drive codingManual tests are error proneTests provide documentation

You know what your system does

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



50

Test Pyramid – OLD modelMost testing was done by the QA team

manually orPlay / record automationThrough the GUI

Based on the V-modelUnit tests were sporadic

Developers knew little about testing

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Automated Test Pyramid - Agile (Mike Cohn)

Unit tests are the base layerProvide the fastest feedbackBest ROI

Middle layerbecomes functional regressiontests

GUI layerMay be partially automatedMostly exploratory testing

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Test CoverageHow are you going to know when you are done?What is enough?Collaborate to decide what coverage you wantThink risk – include the customerFunctional coverageLines of code? … be carefulWhat tools can you use?

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




53

Test Plan

Benefit is in the planningProject specificFocus on what is really neededRefer to a strategy or testing approach docfor static informationShould it be static or dynamic?

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




54

Project Test Planning ToolsQuality Management Strategy

Outlines the consistent test approachAgile Testing QuadrantsAutomated Test PyramidTest Plan Alternatives

Test Plan (simplified)Test WorkbookTest Matrix

Test Coverage Checklist

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



Light-weight Test Plan

Project specifichighlights project critical information

Risks, high level scope, assumptions,constraintsBenefit is in the planningFocus on what is really neededRefer to QM Strategy doc for static infoShould it be static or dynamic?

Sample test plan

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



56

Test MatrixMechanism to think about functionality andpossible test conditionsHigh level view for release planningValue is in the planningValue can also be in the visible progress report

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com



57

Test Plan Matrix – one exampleClic

k to buy N

OW!PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




58

Testing Planning ToolsHeuristics / Mnemonics

Elisabeth HendicksonJonathan Kohl / James Bach (SFDPOT)

Decision tableLogical relationships

Mind mappingUse cases – flow mapping

Cause & effectFocus on userHappy pathOther paths

State transition diagrams

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




59

Planning Your Test Strategy

Consider scope, priorities, risksTools that solve the problemInvolve customersCollaborate with programmersDocument only what is usefulConsider all four quadrantsTest matrix - big picture

allows whole team to understand

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




60

Group Discussion

What quadrants does your team use ‘back home’Which quadrant could your team “back home”improve?What will you do to address this, when you goback?How will you approach your test planning

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




61

Session Retrospective

Use lessons learned to improveReview - are we doing all we could?What would you like to know more of?What was the most useful?

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com




62

“The” Book

Agile Testing: A Practical Guide forTesters and Agile Teams

By Janet Gregory and Lisa Crispin

Available on• Amazon.com• Amazon.ca

www.agiletester.ca

www.janetgregory.ca

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com

http://www.agiletester.ca

http://www.janetgregory.ca




63

ResourcesCollaboration Explained: Jean TabakaTesting Extreme Programming, By Lisa Crispinand Tip HouseFearless Change: Patterns for introducing newideas, Linda Rising and Mary Lynn [email protected] (Brian Marick’s site)www.testobsessed.comwww.satisfice.com

Click t

o buy NOW!

PDF-XChange


k to buy N

OW!PDF-XChange

www.docu-track.com

mailto:[email protected]

http://www.exampler.com(Brian

http://www.testobsessed.com

http://www.satisfice.com



Testing Techniques For Agile Testers - Janet Gregory

Technology

Transcript of Testing Techniques For Agile Testers - Janet Gregory