Testing in Agile Projects:Where Things Stand Today

Brian Marickmarick@testing.com

Copyright © 02004 by Brian Marick. Permission granted to reproduce verbatim.Some images copyright www.arttoday.com.

What Are Agile Projects?

• An attitude toward change

• An attitude toward software

• Some attitudes toward people

The Manifesto for Agile Software Development, www.agilemanifesto.org

Changing Requirements Are Swell

Oh! Let’s add this!

Sure!Now that I see my

feature, I don’t like it. How should wechange it?

Software Can Be Soft

• Programs can become better, cleaner, and more capable

• They become changeable by being successfully changed– not mainly by planning for change

• Frequent new requirements “train” the code and the coders

“Agile methods, the Emersonian worldview, and the dance of agency”www.visibleworkings.com/papers/agile-methods-and-emerson.html

People

• Written documentation is a poor substitute for continuous conversation

• Generalists trump specialists• Teams can self-organize• Trust

– “… if you ask for help, someone has to help you” - Lisa Crispin

Agile Methodologies

• Extreme Programming

• Scrum

• DSDM

• Evolutionary delivery and staged delivery share many characteristics

See “Reading” slides at the end

The Import

Kaner, Bach, and Pettichord, Lessons Learned in Software Testingwww.context-driven-testing.com/wiki/scribble.cgi

These attitudes toward change, software, and people are the context for testing in Agile projects

Four Types of Testing

Geoffrey Moore, Crossing the Chasm, p. 19

Programmer Testing

I need an object that records eachtime segment

Test-driven designUnit testing

Programmer Testing

The test comes first

Programmer Testing

Code is written topass the test

Programmer Testing

Not done? Write another test…

Programmer Testing

And the code to pass it.All earlier tests continue to pass

Programmer Testing

Code awkward? Fix it now.Tests continue to pass

(Refactoring)

Martin Fowler, Refactoring

Programmer Testing

Eventually, the jelly is cooked, nailed down,and ready for further change

Status

Why was this decision made?

I need an object that records eachtime segment

Because of a Customer Representative

…I want to scribble notes aboutwhat I’m doing…

FIT Tests

Makes senseto the

customerrepresentative

Reminder,not

requirement

Easy to run

fit.c2.com

Test Results

Browser-Friendly

TestsFirst

Tests Make Change Smooth

smallchange

smallchange

smallchange

Has anythingbroken?

Has anythingbroken?

Has anythingbroken?

Tests Inform Programmers

• There are no explicit requirements or specifications– so tests cannot check code against them

• Tests serve same goal as requirements or specifications– they provoke programmers to

write the right programNo way!

Requirements for Test Notation

• Provoking the right code

• Improving product conversation– tests are something to talk about– ground conversation in the concrete– forging a common vocabulary

• Making possibilities more noticeable– explaining to someone else supplements

trying out working software– concreteness sparks ideas

The Tester As Participant

How can we best be concrete? What “goes without saying”?

What should the product not do?Who’s being overlooked?What bugs seem likely?

“Tester” As Job Title

How can we best be concrete? What “goes without saying”?

What should the product not do?Who’s being overlooked?What bugs seem likely?

Status

acceptance tests, customer tests, whole-product tests, business-facing tests…

Mid-Course Observation

Business Facing

Technology Facing

These tests primarilysupport programming(as well as the whole

team’s understanding)

Is This Really Testing?

• Checked examples– for discussion– for confident implementation– where are the bugs?

• Change detectors– for confident implementation– where are the bugs?

What About When the Examples Are Bad Examples?

(incomplete, misleading)

Extending the Model

Examples that usebusiness

terminology

Examples that useinterior

terminology

Business Facing

Technology Facing

Su

pp

ort

Pro

gra

mm

ing

Extending the Model

Examples that usebusiness

terminology

Examples that useinterior

terminology

?

?

Business Facing

Technology Facing

Su

pp

ort

Pro

gra

mm

ing

Critiq

ue P

rod

uct

Critiquing the Product

• What resource do we newly have?– the working product, including new code

• Exploratory testing– “simultaneous learning, test design, and

test execution” - James Bach

• Doing what?– diverse users and their scenarios– imaginative end-to-end testing– some opportunistic feature testing

www.satisfice.com/articles.shtml

Exploratory Bug Finding

A Quadrant Entry

Examples that usebusiness

terminology

Examples that useinterior

terminology

User-centeredbug reports

?

Business Facing

Technology Facing

Su

pp

ort

Pro

gra

mm

ing

Critiq

ue P

rod

uct

Status

Still Not Addressed

• What about security bugs, configuration bugs, performance problems, bugs revealed under load, usability problems (like suitability for color-blind people), etc. etc. etc.?– difficult to specify by example– whole-product, but not central to domain

These Are Technology Issues

• Understanding of implementation more important than understanding of a particular domain

Examples that usebusiness

terminology

Examples that useinterior

terminology

User-centeredbug reports

?

Business Facing

Technology Facing

Su

pp

ort

Pro

gra

mm

ingC

ritiqu

e Pro

du

ct

The Good News

My Take on the State of the Practice

Examples that usebusiness

terminology

Examples that useinterior

terminology

User-centeredbug reports

“ility” bugreports

Business Facing

Technology Facing

Su

pp

ort

Pro

gra

mm

ing

Critiq

ue P

rod

uct

Summary: Testing in Agile Projects

Reading (1)• Agile development in general

– www.agilealliance.org

– Agile Software Development, Alistair Cockburn

– “Agile methods, the Emersonian worldview, and the dance of agency”, Brian Marick, www.visibleworkings.com/papers/agile-methods-and-emerson.html

• Extreme Programming– Extreme Programming Explained, Kent Beck

– www.xprogramming.com/xpmag/whatisxp.htm

• Scrum– Agile Software Development with Scrum, Schwaber and Beedle

– www.mountaingoatsoftware.com/scrum

• DSDM– DSDM: Business Focused Development, DSDM Consortium

– www.dsdm.org

Reading (2)

• Agile from a tester’s point of view– www.testing.com/agile– Testing eXtreme Programming, Lisa Crispin and Tip House– agile-testing@yahoogroups.com– www.testing.com/cgi-bin/blog

• Programmer testing– Test-Driven Design by Example, Kent Beck– Test-Driven Development: A Practical Guide, Dave Astels– Pragmatic Unit Testing, Hunt and Thomas– testdrivendevelopment@yahoogroups.com

Reading (3)

• Exploratory testing– www.satisfice.com/articles.shtml– www.testingcraft.com/exploratory.html

• Context-driven testing– Lessons Learned in Software Testing, Kaner, Bach, and

Pettichord– www.context-driven-testing.com/wiki/scribble.cgi

• Miscellaneous– FIT: fit.c2.com (see also fitnesse.org)– Crossing the Chasm, Geoffrey Moore– Refactoring, Martin Fowler (et. al.)