Agile Testing isn’t Risking IT!
description
Transcript of Agile Testing isn’t Risking IT!
Agile Testing isn’t Risking IT!
Wednesday 7 November 2012Eurostar 2012
2© All rights reserved
Agile Testing
isn’t Risking IT!
Introduction
What is it all about?
Which problems did I face?
How to do it right?
Recap-
Q & A
12345
6
3© All rights reserved
Agile Testing
isn’t Risking IT!
Introduction
What is it all about?
Which problems did
I face?
How to do it right?
Recap-
Q & A
12345
6
© All rights reserved
Agile in a nutshell
4
=
ProductBacklog
SprintBacklog
Sprint2-4 wk
Scrum24 u
Product Owner Team Scrum Master
DemoPlanning Meeting
Retrospective
© All rights reserved
“A process or method to analyse the risks to the product, with the intention to utilize the testing effort in the most valuable way.”
Product Risk Analysis in a nutshell
Classification: High
Classification: Medium
Classification: Low
Product RiskRequirement
“The likelihood that a product fails in relation to the conceivable damage when a product fails in production.”
5
6© All rights reserved
Risk & Requirement Based Testing in a nutshell
Risk without requirement:• Add requirement (a fault found
early)• Remove Risk (no worries about
nothing)
Requirement without Risk• Add product risk (improves
coverage of test)• Remove Requirement (only
make what is needed)
Requirements
Product Risks
7© All rights reserved
Agile Testing
isn’t Risking IT!
Introduction
What is it all about?
Which problems did I face?
How to do it right?
Recap-
Q & A
12345
6
© All rights reserved
Risk analysis were not translated to the Backlog which makes it difficult to utilise a risk based approach.
Merging or splitting features invalidate the PRA.
SPECJ
SPECI
SPECH
SPECG
SPECF
SPECE
SPECD
SPECC
SPECB
SPECA
REQ02
REQ06
REQ04
REQ10
REQ09
REQ05
REQ01
REQ08
REQ07
REQ03
?
?
?
8
© All rights reserved
Risk analysis were not updated every iteration, which could mean that they will be out dated within a sprint.
PRA isn’t updated
REQREQREQREQREQREQREQREQREQREQ
PRPR
REQREQREQREQREQREQREQREQREQREQ
PRPRPRPR
REQREQREQREQREQREQREQREQREQREQ
PRPR
PR
PR
9
© All rights reserved
Risk analysis were only done from the requirements which meant that some product risks were missed.
PRA are done with tunnel vision
SpecifyTest Cases
DetermineRisk
OrderRequirements
AnalyseRequirements
Analyse Risk
Analyse Risk
OrderRequirements
AnalyseRequirements
SpecifyTest Cases
10
© All rights reserved
Risk analysis were done, but the outcome wasn’t taken into account in the planning meetings.
Risk class isn’t used in planning meeting
2 SP
4 SP
4 SP
2 SP
2 SP
6 SP
3 SP
2 SP
2 SP
8 SP
2 SP
3 SP
4 SP
6 SP
2 SP
Team Velocity=
± 14 Story Points
1
2
3
42 SP
4 SP
4 SP
2 SP
2 SP
6 SP
3 SP
2 SP
2 SP
8 SP
2 SP
3 SP
4 SP
6 SP
2 SP
1 SP
2 SP
4 SP
2 SP
6 SP
6 SP
9 SP
6 SP
6 SP
8 SP
6 SP
9 SP
2 SP
3 SP
6 SP 1
2
4
6
5
3High = 3 n
Medium = 1 n
Low = ½ n
13
© All rights reserved
Risk analysis were done, but the risks where prioritised using MoSCoW, which doesn’t automatically fit in a Agile environment.
Testing is prioritized based on product risks
=
M
S
W
C=
=
M
S
W
C =
M
S
W
C
Sprint 1 Sprint 2 Sprint 3
12
13© All rights reserved
Agile Testing
isn’t Risking IT!
Introduction
What is it all about?
Which problems did
I face?
How to do it right?
Recap-
Q & A
12345
6
© All rights reserved
Gather Risk Items• Gather & Order Requirements
• Gather Product Risks
Plot Risk Items• Map Requirements
& Product Risks
Calculate Risk Items• Determine Impact
& Likelihood Factors
• Score & Calculate Risk Items/Factors
Strategy to tactic• Risk class tactic• Sprint tactic
Gather Risk Items
14
© All rights reserved
Gather & Order Requirements
Rank #
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
REQ04
REQ02
REQ07
REQ03
REQ01
REQ05
REQ06
REQ08 REQ
09 REQ10
REQ14
REQ11
REQ12
REQ13
REQ15
H
M
LREQ15
REQ14
REQ13
REQ12
REQ11
REQ10
REQ09
REQ08
REQ07
REQ06
REQ05
REQ04
REQ03
REQ02
REQ01
REQ15
REQ14
REQ13
REQ12
REQ11
REQ10
REQ09
REQ08
REQ07
REQ06
REQ05
REQ04
REQ03
REQ02
REQ01
MU
ST
SH
OU
LD
CO
ULD
17
© All rights reserved
Gather Product Risks
PR 02
PR 05
PR 01
PR 04
PR 03
PR 08
PR 07
PR 06
PR 09
16
© All rights reserved
Gather Risk Items• Gather & Order
Requirements• Gather Product
Risks
Plot Risk Items• Map Requirements & Product Risks
Calculate Risk Items• Determine Impact
& Likelihood Factors
• Score & Calculate Risk Items/Factors
Strategy to tactic• Risk class tactic• Sprint tactic
Plot Risk Items
17
18© All rights reserved
Requirements
Map Requirements & Product Risks 1/3
Product Risks
REQ04
PR02
PR05
PR01
PR04
PR03
PR08
PR07
PR06
REQ02
REQ07
REQ03
REQ01
REQ05
REQ06
REQ08 PR
09REQ09 REQ
10
REQ14
REQ11
REQ12
REQ13
REQ15
19© All rights reserved
REQ14
REQ11
REQ12
REQ13
REQ15
Requirements
Map Requirements & Product Risks 2/3
Product Risks
REQ04
PR02
PR05
PR01
PR04
PR03
PR08
PR07
PR06
REQ02
REQ07
REQ03
REQ01
REQ05
REQ06
REQ08 PR
09REQ09 REQ
10
© All rights reserved
Map Requirements & Product Risks 3/3
20
Product Risk
Requirement
PR 01
PR02
PR03
PR04
PR05
PR06
PR07
PR08
PR09
REQ 01 ● ● ● ●
REQ 02 ● ● ●
REQ 03 ●
REQ 04 ● ●
REQ 05 ● ●
REQ 06 ● ●
REQ 07 ● ●
REQ 08 ● ●
REQ 09 ● ●
REQ 10 ● ● ●
REQ 11 ●
REQ 12 ●
REQ 13 ● ● ●
REQ 14 ● ● ●
REQ 15 ● ●
H
M
L
MU
ST
SH
OU
LD
CO
ULD
© All rights reserved
Gather Risk Items• Gather & Order
Requirements• Gather Product
Risks
Plot Risk Items• Map Requirements
& Product Risks
Calculate Risk Items• Determine Impact
& Likelihood Factors
• Score & Calculate Risk Items/Factors
Strategy to tactic• Risk class tactic• Sprint tactic
Calculate Risk Items
21
22© All rights reserved
Impact factors e.g.• Frequency of Use• Visibility• Cost of failure• …
Determine Impact & Likelihood Factors
Likelihood Factors e.g.• Complexity• Re-use• Interfaces• Dependencies• Size• …
Interfaces
None
1 - 2
> 2
Visibility
Internal
Partners
Public
© All rights reserved
Product Risk
Requirement
PR 01
PR02
PR03
PR04
PR05
PR06
PR07
PR08
PR09
REQ 01 ● ● ● ●
REQ 02 ● ● ●
REQ 03 ●
REQ 04 ● ●
REQ 05 ● ●
REQ 06 ● ●
REQ 07 ● ●
REQ 08 ● ●
REQ 09 ● ●
REQ 10 ● ● ●
REQ 11 ●
REQ 12 ●
REQ 13 ● ● ●
REQ 14 ● ● ●
REQ 15 ● ●
Score & Calculate Item/Factor
23
LIKELIHOOD PRODUCTRISK
RISKCLASS
IMPACT
Interfaces
Size Re-use TOTAL TOTAL Users Visibility
Cost of Failure
> 2 > 5 SP None High PR01
Medium Low < 5 Internal < 1k
None 2 – 5 SP Partly Low PR02
Medium Medium > 50 Public < 1k
1 – 2 2 – 5 SP Completely
Medium PR03
Medium Medium 5 - 50 Internal < 1k
2 > 5 SP Partly High PR04
High Medium > 50 Partners
> 10k
2 < 2 SP Completely
Medium PR05
Medium Medium < 5 Partners
< 1k
1 – 2 < 2 SP Completely
Low PR06
Low Low < 5 Internal 1k – 10k
1 – 2 2 – 5 SP Partly Medium PR07
High High 5 - 50 Public > 10k
None 2 – 5 SP None Low PR08
Low Low 5 – 50 Internal < 1k
None < 2 SP Completely
Low PR09
Low Low 5 - 50 Internal < 1k
PR01
PR02
PR03
PR04
PR05
PR06
PR07
PR08
PR09
PR 01
PR 02
PR 03
PR 04
PR 05
PR 06
PR 07
PR 08
PR 09
Product Risk
Requirement
PR 01
PR02
PR03
PR04
PR05
PR06
PR07
PR08
PR09
REQ 01 ? ? ? ?
REQ 02 ? ? ?
REQ 03 ?
REQ 04 ? ?
REQ 05 ? ?
REQ 06 ? ?
REQ 07 ? ?
REQ 08 ? ?
REQ 09 ? ?
REQ 10 ? ? ?
REQ 11 ?
REQ 12 ?
REQ 13 ? ? ?
REQ 14 ? ? ?
REQ 15 ? ?
HMMH
H
H
Product Risk
Requirement
PRX
PRX
PRX
REQ X H M M
REQ X H M L
REQ X M M L
M H M MM M H M M M H M M H M M H L M L M M M H L M L L L LM L
© All rights reserved
Gather Risk Items• Gather & Order
Requirements• Gather Product
Risks
Plot Risk Items• Map Requirements
& Product Risks
Calculate Risk Items• Determine Impact
& Likelihood Factors
• Score & Calculate Risk Items/Factors
Strategy to tactic• Risk class tactic• Sprint tactic
Strategy to tactic
24
© All rights reserved
High Medium Low
Check documentation Inspection Walkthrough Peer review
UT test case review Test lead Tester -
ST test case review Test lead Tester -
Include test case in regression set
Sprint + release
Release -
Automate test case CodedUI Record & Playback
-
Test execution on browsers
Top 3 Top 2 Top 1
… … … …
Risk class tactic
High = Effort * 3 Medium = Effort * 2 Low = Effort * ½
25
© All rights reserved
Sprint Tactic – Scope Sprint 1
26
REQ15
REQ14
REQ11
REQ10
REQ01
REQ09
REQ07
REQ06
REQ02
REQ08
REQ04
REQ03
REQ13
REQ12
REQ05
PR09
PR08
PR07
PR06
PR05
PR04
PR03
PR02
PR01
Product Risk
Requirement
PR02
PR04
PR06
PR08
PR09
REQ 05 H MREQ 12 L REQ 13 M L L
Rank #
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
© All rights reserved
Sprint Tactic – Scope Sprint 2
27
REQ15
REQ14
REQ11
REQ10
REQ01
REQ09
REQ07
REQ06
REQ02
REQ08
REQ04
REQ03
REQ13
REQ12
REQ05
PR09
PR08
PR07
PR06
PR05
PR04
PR03
PR02
PR01
Product Risk
Requirement
PR02
PR03
PR04
PR09
REQ 03 M REQ 04 M M REQ 08 H L
Rank #
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
© All rights reserved 28
Scrum bord
To Do Doing Done
REQ 3
REQ 4
REQ 8
S P R I N T 2
© All rights reserved
Insight into Product Risk & Requirement coverage
29
Product Risk
Requirement
PR01
PR02
PR03
PR04
PR05
PR06
PR07
PR08
PR09
REQ 05 REQ 12 REQ 13 REQ 03 M REQ 04 M M REQ 08 H LREQ 02 M M H REQ 06 M H REQ 07 M M REQ 09 M L REQ 01 M H M MREQ 10 M M M REQ 11 H REQ 14 L LREQ 15 M L
Doin
gTo
do
Product Risk
Requirement
PR01
PR02
PR03
PR04
PR05
PR06
PR07
PR08
PR09
REQ 05 REQ 12 REQ 13 REQ 03 REQ 04 REQ 08 REQ 02 M M H REQ 06 M H REQ 07 M M REQ 09 M L REQ 01 M H M MREQ 10 M M M REQ 11 H REQ 14 L LREQ 15 M L
Don
eD
oin
gTo
doProduct Risk
Requirement
PR01
PR02
PR03
PR04
PR05
PR06
PR07
PR08
PR09
REQ 05 REQ 12 REQ 13 REQ 03 REQ 04 REQ 08 REQ 02 REQ 06 REQ 07 REQ 09 REQ 01 M H M MREQ 10 M M M REQ 11 H REQ 14 L LREQ 15 M L
Don
eD
oin
gTo
do
Product Risk
Requirement
PR01
PR02
PR03
PR04
PR05
PR06
PR07
PR08
PR09
REQ 05 REQ 12 REQ 13 REQ 03 REQ 04 REQ 08 REQ 02 REQ 06 REQ 07 REQ 09 REQ 01 REQ 10 REQ 11 REQ 14 L LREQ 15 M L
Don
eD
oin
gTo
do
Product Risk
Requirement
PR01
PR02
PR03
PR04
PR05
PR06
PR07
PR08
PR09
REQ 05 REQ 12 REQ 13 REQ 03 REQ 04 REQ 08 REQ 02 REQ 06 REQ 07 REQ 09 REQ 01 REQ 10 REQ 11 REQ 14 REQ 15
Don
eD
oin
g
© All rights reserved
Recap
Agile RRBT• Gather Risk Items• Plot relationship between Risk Items• Calculate and Classify Risk Items• Differentiate the work
Lessons Learned• Analyses can only be done with the
backlog in place.• Plan to iterate and evolve your
analyses in sync with the sprints.• Don’t use the backlog as your only
starting point for your analyses.• The result of your analyses is essential
in the planning meeting.• Order your testing effort in the small.
30
B.W.F.P.M. BRONNEBERGTEST MANAGER / CONSULTANT
REQUIREMENT & QUALITY MANAGEMENT
WWW.BRAMBRONNEBERG.NL
@BRAMBRONNEBERG
+31 6 55 11 35 10