Agile Days China

Evolution of Agile Testing (A tester's view)

Erik Petersenecp@testingspot.netTwitter: erik_petersen

Tech talk: http://bit.ly/80-20-rules

Familiar faces?

Methodologists meet in the snow

Mid February 2001, 17 people met in the Utah mountains over a weekend

They represented styles of lightweight software development

Only one, Brian Marick, had major interest in testing (a quiet person)

Another tester invitee, James Bach, could not attend (a loud person)

Non-light manifesto Individuals & interactions over processes &

tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following plan

Value in all, but items on left more valuable

Where's testing? Neither the agile manifesto or supporting

principles mention testing, focussing on ”sponsor, developer, user”

Principles do mention ”architecture, requirements & designs” but no testing or validation activity mentioned explicitly

Agile testing V1.0 Strong Extreme Programming slant

Developers do all tests, using TDD for design & automated unit tests, Junit mostly

No need for system tester role, ”all tests automated”

Developers to learn extra testing skills from testers then replace them

Kent Beck tells conference of testers they will all be out of jobs in a few years

Agile dev view

Agile tester view

A week of Ruby dev, test & talk Mid June 2003, a group of agile devs and

exploratory testers spend a week together talking and creating software at ”Agile Fusion” (Erik watches via webcam!)

Early version of Ruby seen as immature by devs (but great language for testing)

Some agile devs understand importance of exploratory testing, e.g Ward Cunningham

Agile Testing manifesto?

”It’s very important to understand that we in this Agile Fusion conference did not come to agreement on *any* specifically phrased idea about what agile testing is or isn’t. We tried to discuss it, but it quickly became apparent that

there are important philosophical & terminological differences among us.”

James Bach

“Important differences”?

Agile testing V2.0 Strong XP Programming slant weakening

Regression testing done by developers, using automated unit tests, now Xunit

Some appreciation for system tester role, e,g Thoughtworks, Mary Poppendieck

Some developers learn more about testing and importance of exploratory testing

Agile testing champions, e.g Elisabeth Hendrickson

Tools of the trade

Enter the test toolsmith Developer explicitly supporting less

technical testers, or more busy devs

Creating tools to support testability

Automating data creation, configuration table setup, security profiles, UI information fields only visible in test mode or hidden in html, etc

Focussed on manual & automated tests

Testability champion, Brett Pettichord

Test tools rule Rapid expansion of free functional tools

Framework for Integrated Test (FIT) 2002, java based implementation of customer examples as tests, lead dev Ward Cunningham

Selenium 2004, web application driver using javascript, lead dev Jason Huggins

Many other less popular tools

What does this mean?

Dependency Injection 1 What does this mean?

Imagine you run a window washers collective. Each worker has their own basic gear, buckets, sponges, scrapers, and can handle smaller jobs unassisted. Larger jobs need ropes, scaffolding or cherry pickers, so they need to be provided for the workers in those cases. (continued)

Dependency Injection 2 Dependency injection provides complicated

material to a worker function to help it do its job. This simplifies the design, and makes it more flexible allowing different types of materials to be provided in different cases. Another advantage is the ability to provide fake materials, when we are only interested in testing the function to check it still works properly.

From www.testingspot.net (Erik's link site)

Agile testing V3.0 Flavors Regression testing by developers, using

TDD & automated unit tests, with dependency injection & testability

Testsmiths create custom tools for their teams

Functional automation by developers or technical testers using tools like Selenium

Customer examples linked by dev tester to fixtures create simple tests, like FIT

ET in Agile Development Jim Shore & Chromatic write ”Art of Agile

Development”, 2007 with community review of each chapter

Lone voice of tester reviewer in dev cloud (Erik), pushing tester role & viewpoint

Book expanded to include ET Viewpoint: ET needed to find bugs missed by devs (to improve process to find next time!)

Included Exploratory testing chapter written by Elisabeth Hendrickson

Different stylistic views• 1) Does a agile team use a issue tracker?

Against: don't document just discuss & fixFor: bug history is valuable for bug clusters, empower user to raise issues or concerns

• 2) Is quality more important than velocity?Against: need to always deliver value over fixesFor: bugs may hide other bugs, more (potential) technical debt

Agile testing for testers & team Agile Testing stage at Agile 2009

conference

Exploratory testing sessions at Agile conference, Erik Petersen 2008, 2009, Jon Bach 2010

”Agile Testing” 2009, by Crispin & Gregory gets great reviews across all agile tribes

Strong agile tester community

Bughunts becoming popular in industry

Joining the cause

The 2nd wave of test tools Cucumber 2009, english-like fixture based

tool dominating functional test automation, lead developer Aslak Hellesoy

Selenium and Selenium Grid almost industry standard

Domain Specific Languages (e.g in Groovy), Robot framework, etc

AA functional test tools group discussing and leveraging tools, also GTAC (Google)

Key (system) testing skills Test techniques: e.g. boundary analysis,

equivalence partitioning, test intuition

Test modelling: domain, application, navigation, authorisation, etc.

Test case design and coverage

Defect management

Failure modelling (often innate skill)

Shadow architect & designer/ devils advocate

Key skill: Exploratory testing ET Framework: Three main parts

Investigation Wide across app, or deep to feature

Exploration Confirming app & other models, or trying

failure models

Reflection Maintain focus or change to next priority Investigate bug clusters

Attempt to make implicit skill visible

Agile testing today Still no agile testing manifesto, just many

flavors (like with agile)

New tools can be adopted quickly worldwide, e.g Cucumber

Many great developer testers but still specialist tester role needed

Risk based exploratory testing preferred technique for rapid testing

TDD ”checks” versus human ET ”tests”

Together (mostly)

Evolution of Agile Testing (A tester's view)

Erik Petersenecp@testingspot.netTwitter: erik_petersen

Tech talk: http://bit.ly/80-20-rules

Selected ReferencesTestability for Automation -Dave Catlett

presentation, 2004

http://www.nwcpp.org/Downloads/2004/ImprovingTestabilityNWCPP.ppt

Includes link to Brett Pettichord paper

AgileAlliance.org

Testingspot.net – Erik’s links site

Star Wars Posters – Cliff Chiang

Vader Portrait – Greg Peltz