Tricentis Webinar: Mastering Your Business Risk

© 2016 by© 2016 by .

Mastering your Business RiskElmar Pauwels

© 2016 by

Acceptance

Deployment

Waterfall

Requirements

Design

Implementation

Testing

Oops, Hello Mr. Bugs…!

eternally long∞

delivery cycle time𝟎

Evolutionary Testing Methodologies

© 2016 by

∞ 𝟎shortish

DEV DEV

OPS OPS

DEV DEV DEV

It compiles, it works on my machine and therefore it works!

a little less a lot lesseternally long

delivery cycle time

OPS → OOPS!

Agile

Siloisation


© 2016 by

DevOps

∞ 𝟎close to zero

Dev Ops

ContinuousRelease & DeploymentCollaborative

Development

ContinuousTesting

ContinuousMonitoring

ContinuousIntegration

ContinuousFeedback

Today!

Tomorrow!?

InsaneSpeed

SpookyAction

shortisha little less a lot lesseternally long

delivery cycle time

…our conclusion

…his conclusion


© 2016 by

Enterprise system landscapes are alike disease gene networks

ContinuousTesting

∞ 𝟎close to zero

InsaneSpeed

SpookyAction

shortisha little less a lot lesseternally long

delivery cycle time

…our conclusion

…his conclusion

HolyTestingGrail

?Complexity Quantity VarietySpeed

Today!

Tomorrow!?

Challenges

It’s not equal to automation!

It is about

How to keep the pace?

© 2016 by

As a tester I strive after continuous testing in order to provide instant feedback about the quality of the

product at any time.*Being able to execute the maximum amount of the most

important test cases automatically on demand to remove as much as possible business risk earliest possible.

What is continuous testing?

© 2016 by

Now

Then

Test

Cas

es

Auto

mat

ion

Leve

l

Risk Coverage

High GoalOriented

ValueOriented

QualityOriented

ProcessOriented

TimeOriented

EffortOrientedHigh

Low

Low

Continuous Testing

Overall Goal

© 2016 by

Leave your shoesbehind and drive.

Automate

A

B

ManageKeep the traffic

lights green.

1

Why isn’t testing just about speed?

OptimizeFind the shortest

possible path.

right

right

do things

do the

things

Effectivity

Efficiency

GoalOriented

ValueOriented

QualityOriented

ProcessOriented

TimeOriented

EffortOriented

© 2016 by

Passed

Failed

Not Executed

Leak

Business Risk

Passed

Failed

Not Executed

Test Cases

Measuring quality in terms of risk is to know, i.e. if you can‘t measure it, you can‘t improve it.

scrutinizing test reportsBad Good

From Counting Test Cases to Risk Coverage

© 2016 by

Risk quantifies the potential oflosing something of value.

*That’s our axiom. A premise or starting point of reasoning. It’s a premise so evident as to be accepted as true without

controversy.

© 2016 by

Securities TradingCapture Order

Client Side Validation

Market Side Validation

Check EligibilityCheck SuitabilityCheck Availability

Rectify OrderCancel Order

256

AbsoluteWeight

512

1024

64

64

1024

32

32

4 4

5 4

5 5

3 3

3 3

5 5

2 3

1 4

DamageClass

FrequencyClass

It‘s all about relative comparisons!

© 2016 by

256

AbsoluteWeight

512

1024

64

64

1024

32

32

80% 80%

26.7% 33.3%

23.7% 88.8%

1.5% 5.6%

1.5% 5.6%

53.3% 66.7%

10% 10%

10% 10%

RelativeWeight

Business RiskContribution






A natural consequence

© 2016 by

80%

26.7%

23.7%

1.5%

1.5%

53.3%

10%

10%

Business RiskContribution






39 12 18 31

60 2 6 32

33 17 50

100

42 21 37

34 5016

1025 25 40

18 22 3030

41 6 9 45

Business RiskCoverage %

The peak of perfection!

© 2016 by

LOWRISK

test

cas

es

N-2 N-1 N N+1 N+2 sprint

critical limit

90%

busi

nes

s ri

sk

75%20%HIGH

RISK

MEDIUM

RISK

…the time needed for testing is infinitely larger than the time available – C. Kaner.

Practical significance

© 2016 by

Busi

ness

Ris

k Co

vera

ge

60%

80%

100%

40%

20%

0% 0

Def

ect

Rate

max

100%

Test Cases20%0% 40% 60% 80%

Critical Defects

𝟏 Weight𝟏𝟏𝟏 𝟏𝟏𝟏… …

A value neutral approach to testing

© 2016 by

Randomness rules!No risk assessed, critical defects are just chance hits.

No characteristics!No plan about what drives the test, selective testing is impossible.

Nothing to learn!No chance to improve anyhow.

A value neutral approach to testing

© 2016 by

60%

80%

100%

40%

20%

0%Busi

ness

Ris

k Co

vera

ge

100%20%0% 40% 60% 80%

Test Cases

Critical Defects0

Def

ect

Rate

max

𝒘𝟏𝒘𝟐𝒘𝟑𝒘 𝟒 𝒘 𝒊 𝒘𝑵… … … … 𝒘 𝒊≥𝒘 𝒊+𝟏Weight

The first steps towards risk-based testing

© 2016 by

Know what matters most!Good knowledge about high-risk areas.

Know what doesn’t matter!Good knowledge about low-risk areas.

Still a lot to learn!Where to reassess risk? Consider defect density!

The first steps towards risk-based testing

© 2016 by

Scrutinize, re-evaluate & improve your risk-based approach

continuously.

*Strive after continuous improvement at defined checkpoints, instead of ultimate perfection. Your product is an ever changing entity, and so do

your risks?

D‘oh, risk-based testing contains risks!

© 2016 by

60%

80%

100%

40%

20%

0%

Busin

ess R

isk C

over

age

100%20%0% 40% 60% 80%

Test Cases

critical defects0

Def

ect

Rate

max

𝒘𝟏𝒘𝟐𝒘𝟑𝒘 𝟒 𝒘 𝒊 𝒘𝑵… … … … 𝒘 𝒊≥𝒘 𝒊+𝟏Weight

HighRisk

do a lot with a little

LowRisk

Testing everything without testing everything

© 2016 by

Doing a lot with a little!Achieve maximum risk coverage by minimum amount of test cases.

Be on the safe side!Learn continuously to keep the critical areas up to date.

Work as a brand protector!Remove as much as possible risk earliest possible.

Testing everything without testing everything

© 2016 by 22

Thank You for your attention

Host: Elmar PauwelsEmail: [email protected]

Follow us on Twitter: @tricentis

Stay in touch!

Tricentis Webinar: Mastering Your Business Risk

Technology

Transcript of Tricentis Webinar: Mastering Your Business Risk