SCRUM & TestingSCRUM & Testing“Back to the Future”“Back to the Future”

Erik van Veenendaal

www.erikvanveendaal.nl

Erik van Veenendaal

� Founder and major shareholder ImproveQS

� In testing since 1989 working for many different clients and in many different roles

� Author “TMap”, “The Testing Practitioner” and many other books and papers

Improve Quality Services BV 2

other books and papers

� Vice-President International Software Testing Qualifications Board (ISTQB) 2005 - 2008

� Vice-Chair TMMi Foundation

� Keynote speaker, e.g. EuroSTAR, STAReast

� Winner of the European Testing Excellence Award (2007)

Improve Quality Services BV

� Service organisation in the area ofTesting, Requirements Engineering and Quality Management

� Consultancy, Subcontracting and Training

www. improveqs.nl

Improve Quality Services BV 3

� Consultancy, Subcontracting and Training

SW Process ImprovementQuality AssuranceIT-AuditingRequirements Engineering& management (IREB)

Testing (TMap, TMMi)Test Process ImprovementCertification (ISTQB)- incl. Advanced !!Inspections / Reviews

What is Agile?What is Agile?

Improve Quality Services BV 4

Why Agile?Why Agile?

� Deliver business value earlier

� Divide and conquer

� The requirements problem

Minimized scope6% 5%Reliable EstimatesFormal methodology

Only 28% of IT projectsOnly 28% of IT projectsis successful !!is successful !!

AGILEAGILE

REQ ENGREQ ENG

Improve Quality Services BV 5

Clear business objectives

16%User involvement

6%

Minimized scope10%

Firm basic requirements

Executive support

12%18%

14%

8%6% 5%

5%Experienced Project Manager

Standard software infrastructure

Other

REQ ENGREQ ENG

Agile Development ModelAgile Development Model

Iteration

High level productrequirements

Improve Quality Services BV 6

High leveliteration

requirements

Product

Design

Implementation & unit testing

Acceptance testing

Documentation

Acceptance

…

“Definition of ‘Done”

Testing ChallengesTesting Challenges

Obvious for agile (iterative) development:

� Regression testing, Test automation

� Quality of specification, Traceability, Time-boxing

Less obvious to most “experts”:

It’s simple but not easyIt’s simple but not easy

Improve Quality Services BV 7

Less obvious to most “experts”:

� Knowledge and skills of test professionals

� The (new) role of developers

� The involvement of the business

� Management awareness

� Test levels and test types

Other Test Levels?Other Test Levels?

Improve Quality Services BV 8

What is agile testing?What is agile testing?

• Testing in an immature (test) organization• The great excuse !!

• Testing in an agile development project

Improve Quality Services BV 9

• Testing in an agile development project• Re-shaping traditional practices

• Testing following the statutes of the agile manifesto

• A different mind set to start from

Agile ManifestoAgile Manifesto

� Individuals & interactions over Processes and Tools

� Working Software over Comprehensive Documentation

� Customer Collaboration over Contract Negotiation

Improve Quality Services BV 10

� Responding to Change over Following a Plan

� Agile Values (Kent Beck)

�Communication, Feedback

�Simplicity, Courage

Core Agile Testing PracticesCore Agile Testing Practices

� Risk based: Product Risk Analysis� communication, customer collaboration, working

software & responding to change

� Reviews: Walkthroughs & Informal Reviews� communication & feedback, customer collaboration

Improve Quality Services BV 11

� communication & feedback, customer collaboration

� Test strategy: (automated) unit testing & regression testing� working software over comprehensive documentation

� Test design: Use informal techniques formally� responding to change, customer collaboration &

no comprehensive documentation

� People: Build experienced and skilled testers� individuals and interactions over processes and tools

The Product Risk MatrixThe Product Risk Matrix

IIIIII

HH

MM

xxx

Must TestCould TestTest design techniquesReview designSupport module testsReview module tests

Exploratory TestingSupport module testsReview module tests

Improve Quality Services BV 12

IVIVIIIIII

LL

MM

LL HMM

x

x

xLikelihood

Impact

Should Test“Won’t Test” Test design techniquesReview design

Exploratory Testing

Exploratory TestingExploratory Testing

� Simultaneous exploration, test design and execution

� Finding the most important defects in the time available

� What about …………

- Test charters available and reviewed

PRODUCTPRODUCT

tests

Improve Quality Services BV 13

- Checklist with most common defects (heuristics)

- Experienced testers (test design techniques)

- Working in pairs during test sessions

- Test logs delivered

- Daily debriefing sessions

- Feedback loop; new risk areas

People SkillsPeople Skills

Test knowledgeTest knowledge-- Test managementTest management

-- TechniquesTechniques

IT knowledgeIT knowledge-- Software developmentSoftware development

-- Requirements Requirements (IREB)(IREB)

via (exchanging) via (exchanging) Practical Practical ExperiencesExperiences, , CoachingCoachingand (formal) and (formal) TrainingTraining

Improve Quality Services BV 14

-- TechniquesTechniques

-- Tools, etc.Tools, etc.

-- Requirements Requirements (IREB)(IREB)

-- Configuration mgt.Configuration mgt.

DomainDomainknowledgeknowledge-- Business processBusiness process

-- User characteristicsUser characteristics

Soft skillsSoft skills-- CommunicationCommunication

-- Critical mindsetCritical mindset

-- Team playerTeam player

SCRUM in Five MinutesSCRUM in Five Minutes

� Sprint – a focused effort for a 30-day period towards fixed goals

� Product Owner – responsible for compiling and maintaining a prioritized product backlog

Improve Quality Services BV 15

and maintaining a prioritized product backlog(a to do list)

� Sprint backlog – to do list for Sprint

� Scrum team – self organized and jointly responsible (5 - 9 people)

� Scrum Master – coaches the scrum team and creates the best possible circumstances

SCRUM ProcessSCRUM ProcessUnderstand requirements

Estimation sessionsHigh level test design

Product risk analysis (test plan)Definition-of-done

Review, DesignTest executionReport, Adjust

Coach, SupportDefine fit criteriaDaily SCRUM

Participate RetrospectiveSystem-integration-test?

End-to-end test?Non-functional tests?

Improve Quality Services BV 16

Review requirements

Experiences Experiences -- Finance DomainFinance Domain

� Domain knowledge extremely important

� Manage on process, not functionality

� Exploratory testing & also

Problem areas

� Product quality hard to measure

� Watch-out for the V-model

� Task not well split-up

Improve Quality Services BV 17

� Exploratory testing & alsotest design techniques

� Automated regression testing (QTP)

� Never a dull moment

� Great to work in a team

� Belbin assessments

� Task not well split-up

� Business will try to squeeze in one more CR

� Continuous time pressure

� Time-to market is not 4 weeks

Experiences Experiences –– Medical DomainMedical Domain

� Product Risk Analysis

� Detailed definition-of-done

� Pair programming

� Unit tests & code coverage

Problem areas

� Non-Functional Testing

� Synchronization

� How much documentation?

Improve Quality Services BV 18

� Exploratory testing

� Automated regression test

� Weekly product demo

� Test levels outside sprint

� Empowerment

� Retrospective meetings

documentation?

Experiences Experiences –– Embedded SWEmbedded SW

� Product Risk Analysis

� Detailed definition-of-done

�Agile Master Test Plan

� Pair programming

Problem areas

� Testers profile

� Documentation

� Coherent SCRUM and other testing

Improve Quality Services BV 19

� Unit tests & code coverage

� Exploratory testing

� Automated regression test

�No changes last 4 days

� Test levels outside sprint

�Design Reviews

other testing

� Works great for SW, but …

� Reliability

The Agile TesterThe Agile Tester

� Focus is on delivering business value

� Provides a skeptical view of the quality of the system

� Does more than “just” test

Improve Quality Services BV 20

� Does more than “just” test

� Also performs test management activities

� Great communicator, coach and team player

� Has courage to get out of his comfort zone

� What about the test manager?

ProcessProcess� Define an overall test strategy� Define an approach to development (unit) testingPeoplePeople

blueblue--printprint

How to start?How to start?

Improve Quality Services BV 21

PeoplePeople� Define role and develop skills of SCRUM testers� Change mindset and develop testing skills of

developersTest automationTest automation� “Only” part of phase two ….

Finally . . .Finally . . .

�Future: A different process and mindset

�Back: Testing is not as easy as it seems (again underestimated, pffffff ..)

Improve Quality Services BV 22

�Study theory, but listen to practitioners

�Testing = Tester

�Professional testers can re-use their knowledge and skills

XXXX

Agile HumorAgile Humor

Improve Quality Services BV 23

Thank You !!Thank You !!