Agile testing isn't risking it! published

Agile Testing isn’t Risking IT!

Wednesday 7 November 2012Eurostar 2012

2© All rights reserved

Agile Testing

isn’t Risking IT!

Introduction

What is it all about?

Which problems did I face?

How to do it right?

Recap-

Q & A

1 2345

6


Agile Testing

isn’t Risking IT!

Introduction


Which problems did

I face?

How to do it right?

Recap-

Q & A

1 2345

6

© All rights reserved

Agile in a nutshell

4

=

ProductBacklog

SprintBacklog

Sprint2-4 wk

Scrum24 u

Product Owner Team Scrum Master

DemoPlanning Meeting

Retrospective


“A process or method to analyse the risks to the product, with the intention to utilize the testing effort in the most valuable way.”

Product Risk Analysis in a nutshell

Classification: High

Classification: Medium

Classification: Low

Product RiskRequirement

“The likelihood that a product fails in relation to the conceivable damage when a product fails in production.”

5


Risk & Requirement Based Testing in a nutshell

Risk without requirement:• Add requirement (a fault found

early)• Remove Risk (no worries about

nothing)

Requirement without Risk• Add product risk (improves

coverage of test)• Remove Requirement (only

make what is needed)

Requirements

Product Risks


Agile Testing

isn’t Risking IT!

Introduction


Which problems did I face?

How to do it right?

Recap-

Q & A

1 2345

6


Risk analysis were not translated to the Backlog which makes it difficult to utilise a risk based approach.

Merging or splitting features invalidate the PRA.

SPECJ

SPECI

SPECH

SPECG

SPECF

SPECE

SPECD

SPECC

SPECB

SPECA

REQ02

REQ06

REQ04

REQ10

REQ09

REQ05

REQ01

REQ08

REQ07

REQ03

?

?

?

8


Risk analysis were not updated every iteration, which could mean that they will be out dated within a sprint.

PRA isn’t updated

REQREQREQREQREQREQREQREQREQREQ

PRPR


PRPRPRPR


PRPRPR

PR

9


Risk analysis were only done from the requirements which meant that some product risks were missed.

PRA are done with tunnel vision

SpecifyTest Cases

DetermineRisk

OrderRequirements

AnalyseRequirements Analyse Risk

Analyse Risk

OrderRequirements

AnalyseRequirements

SpecifyTest Cases

10


Risk analysis were done, but the outcome wasn’t taken into account in the planning meetings.

Risk class isn’t used in planning meeting

2 SP4 SP4 SP2 SP2 SP6 SP3 SP2 SP2 SP8 SP2 SP3 SP4 SP6 SP2 SP

Team Velocity=

± 14 Story Points

1

2

3

42 SP4 SP4 SP2 SP2 SP6 SP3 SP2 SP2 SP8 SP2 SP3 SP4 SP6 SP2 SP

1 SP2 SP4 SP2 SP6 SP6 SP9 SP6 SP6 SP8 SP6 SP9 SP2 SP3 SP6 SP 1

2

4

6

5

3High = 3 n

Medium = 1 n

Low = ½ n

13


Risk analysis were done, but the risks where prioritised using MoSCoW, which doesn’t automatically fit in a Agile environment.

Testing is prioritized based on product risks

=

M

S

W

C=

=

M

S

W

C =

M

S

W

C

Sprint 1 Sprint 2 Sprint 3

12


Agile Testing

isn’t Risking IT!

Introduction


Which problems did

I face?

How to do it right?

Recap-

Q & A

1 2345

6


Gather Risk Items• Gather & Order

Requirements• Gather Product

Risks

Plot Risk Items• Map

Requirements & Product Risks

Calculate Risk Items• Determine

Impact & Likelihood Factors

• Score & Calculate Risk Items/Factors

Strategy to tactic• Risk class tactic• Sprint tactic

Gather Risk Items

14


Gather & Order RequirementsRank #

123456789

101112131415

REQ04

REQ02

REQ07

REQ03

REQ01

REQ05

REQ06

REQ08 REQ

09 REQ10

REQ14

REQ11

REQ12

REQ13

REQ15

H

M

LREQ15

REQ14

REQ13

REQ12

REQ11

REQ10

REQ09

REQ08

REQ07

REQ06

REQ05

REQ04

REQ03

REQ02

REQ01

REQ15

REQ14

REQ13

REQ12

REQ11

REQ10

REQ09

REQ08

REQ07

REQ06

REQ05

REQ04

REQ03

REQ02

REQ01

MU

STSH

OU

LDCO

ULD

17


Gather Product Risks

PR 02

PR 05

PR 01

PR 04

PR 03

PR 08

PR 07

PR 06

PR 09

16




Risks







Plot Risk Items

17


Requirements

Map Requirements & Product Risks 1/3

Product Risks

REQ04

PR02

PR05

PR01

PR04

PR03

PR08

PR07

PR06

REQ02

REQ07

REQ03

REQ01

REQ05

REQ06

REQ08 PR

09REQ09 REQ

10

REQ14

REQ11

REQ12

REQ13

REQ15


REQ14

REQ11

REQ12

REQ13

REQ15

Requirements


Product Risks

REQ04

PR02

PR05

PR01

PR04

PR03

PR08

PR07

PR06

REQ02

REQ07

REQ03

REQ01

REQ05

REQ06

REQ08 PR

09REQ09 REQ

10



20

Product Risk

RequirementPR 01

PR02

PR03

PR04

PR05

PR06

PR07

PR08

PR09

REQ 01 ● ● ● ●REQ 02 ● ● ●REQ 03 ●REQ 04 ● ●REQ 05 ● ●REQ 06 ● ●REQ 07 ● ●REQ 08 ● ●REQ 09 ● ●REQ 10 ● ● ●REQ 11 ●REQ 12 ●REQ 13 ● ● ●REQ 14 ● ● ●REQ 15 ● ●

H

M

L

MU

STSH

OU

LDCO

ULD




Risks







Calculate Risk Items

21


Impact factors e.g.• Frequency of Use• Visibility• Cost of failure• …

Determine Impact & Likelihood Factors

Likelihood Factors e.g.• Complexity• Re-use• Interfaces• Dependencies• Size• …

InterfacesNone1 - 2> 2

VisibilityInternalPartnersPublic


Product Risk

RequirementPR 01

PR02

PR03

PR04

PR05

PR06

PR07

PR08

PR09

REQ 01 ● ● ● ●REQ 02 ● ● ●REQ 03 ●REQ 04 ● ●REQ 05 ● ●REQ 06 ● ●REQ 07 ● ●REQ 08 ● ●REQ 09 ● ●REQ 10 ● ● ●REQ 11 ●REQ 12 ●REQ 13 ● ● ●REQ 14 ● ● ●REQ 15 ● ●

Score & Calculate Item/Factor

23

LIKELIHOOD PRODUCTRISK

RISKCLASS

IMPACTInterfac

esSize Re-use TOTAL TOTAL Users Visibilit

yCost of Failure

> 2 > 5 SP None High PR01

Medium Low < 5 Internal < 1k

None 2 – 5 SP Partly Low PR02

Medium Medium > 50 Public < 1k

1 – 2 2 – 5 SP Completely

Medium PR03

Medium Medium 5 - 50 Internal < 1k

2 > 5 SP Partly High PR04

High Medium > 50 Partners

> 10k

2 < 2 SP Completely

Medium PR05

Medium Medium < 5 Partners

< 1k

1 – 2 < 2 SP Completely

Low PR06

Low Low < 5 Internal 1k – 10k

1 – 2 2 – 5 SP Partly Medium PR07

High High 5 - 50 Public > 10k

None 2 – 5 SP None Low PR08

Low Low 5 – 50 Internal < 1k

None < 2 SP Completely

Low PR09

Low Low 5 - 50 Internal < 1k

PR01PR02PR03PR04PR05PR06PR07PR08PR09

PR 01PR 02PR 03PR 04PR 05PR 06PR 07PR 08PR 09

Product Risk

RequirementPR 01

PR02

PR03

PR04

PR05

PR06

PR07

PR08

PR09

REQ 01 ? ? ? ?REQ 02 ? ? ?REQ 03 ?REQ 04 ? ?REQ 05 ? ?REQ 06 ? ?REQ 07 ? ?REQ 08 ? ?REQ 09 ? ?REQ 10 ? ? ?REQ 11 ?REQ 12 ?REQ 13 ? ? ?REQ 14 ? ? ?REQ 15 ? ?

HMMH

H

H

Product Risk

RequirementPRX

PRX

PRX

REQ X H M MREQ X H M LREQ X M M L

M H M MM M H M M M H M M H M M H L M L M M M H L M L L L LM L




Risks






Strategy to tactic• Risk class tactic

• Sprint tactic

Strategy to tactic

24


High Medium LowCheck documentation Inspection Walkthrough Peer reviewUT test case review Test lead Tester -ST test case review Test lead Tester -Include test case in regression set

Sprint + release

Release -

Automate test case CodedUI Record & Playback

-

Test execution on browsers

Top 3 Top 2 Top 1

… … … …

Risk class tactic

High = Effort * 3 Medium = Effort * 2 Low = Effort * ½

25


Sprint Tactic – Scope Sprint 1

26

REQ15

REQ14

REQ11

REQ10

REQ01

REQ09

REQ07

REQ06

REQ02

REQ08

REQ04

REQ03

REQ13

REQ12

REQ05

PR09

PR08

PR07

PR06

PR05

PR04

PR03

PR02

PR01

Product Risk

RequirementPR02

PR04

PR06

PR08

PR09

REQ 05 H MREQ 12 L REQ 13 M L L

Rank #123456789101112131415


Sprint Tactic – Scope Sprint 2

27

REQ15

REQ14

REQ11

REQ10

REQ01

REQ09

REQ07

REQ06

REQ02

REQ08

REQ04

REQ03

REQ13

REQ12

REQ05

PR09

PR08

PR07

PR06

PR05

PR04

PR03

PR02

PR01

Product Risk

RequirementPR02

PR03

PR04

PR09

REQ 03 M REQ 04 M M REQ 08 H L

Rank #123456789101112131415


Insight into Product Risk & Requirement coverage

29

Product Risk

RequirementPR01

PR02

PR03

PR04

PR05

PR06

PR07

PR08

PR09

REQ 05 REQ 12 REQ 13 REQ 03 M REQ 04 M M REQ 08 H LREQ 02 M M H REQ 06 M H REQ 07 M M REQ 09 M L REQ 01 M H M MREQ 10 M M M REQ 11 H REQ 14 L LREQ 15 M L

Doi

ngTo

do

Product Risk

RequirementPR01

PR02

PR03

PR04

PR05

PR06

PR07

PR08

PR09

REQ 05 REQ 12 REQ 13 REQ 03 REQ 04 REQ 08 REQ 02 M M H REQ 06 M H REQ 07 M M REQ 09 M L REQ 01 M H M MREQ 10 M M M REQ 11 H REQ 14 L LREQ 15 M L

Don

eD

oing

To d

oProduct Risk

RequirementPR01

PR02

PR03

PR04

PR05

PR06

PR07

PR08

PR09

REQ 05 REQ 12 REQ 13 REQ 03 REQ 04 REQ 08 REQ 02 REQ 06 REQ 07 REQ 09 REQ 01 M H M MREQ 10 M M M REQ 11 H REQ 14 L LREQ 15 M L

Don

eD

oing

To d

oProduct Risk

RequirementPR01

PR02

PR03

PR04

PR05

PR06

PR07

PR08

PR09

REQ 05 REQ 12 REQ 13 REQ 03 REQ 04 REQ 08 REQ 02 REQ 06 REQ 07 REQ 09 REQ 01 REQ 10 REQ 11 REQ 14 L LREQ 15 M L

Don

eD

oing

To d

oProduct Risk

RequirementPR01

PR02

PR03

PR04

PR05

PR06

PR07

PR08

PR09

REQ 05 REQ 12 REQ 13 REQ 03 REQ 04 REQ 08 REQ 02 REQ 06 REQ 07 REQ 09 REQ 01 REQ 10 REQ 11 REQ 14 REQ 15

Don

eD

oing


Recap

Agile RRBT• Gather Risk Items• Plot relationship between Risk Items• Calculate and Classify Risk Items• Differentiate the work

Lessons Learned• Analyses can only be done with the

backlog in place.• Plan to iterate and evolve your

analyses in sync with the sprints.• Don’t use the backlog as your only

starting point for your analyses.• The result of your analyses is essential

in the planning meeting.• Order your testing effort in the small.

30

B.W.F.P.M. BRONNEBERGTEST MANAGER / CONSULTANTREQUIREMENT & QUALITY MANAGEMENT

[email protected]

@BRAMBRONNEBERG+31 6 55 11 35 10

Agile testing isn't risking it! published

Documents

Transcript of Agile testing isn't risking it! published