Non Functional Testing

in Agile

Agile Testing Alliance Global

Gathering

Nov.

7S R V Subrahmaniam

PMP, ACP, CSP

Principal Consultant Agile

Altimetrik

Slide 2

Let us play the Tank Game!

Let us build a Tank Iteratively

We, in the next 30 minutes, look at building a

new version of Main Battle Tank Arjan

New set of features

Assume a tank is already in place, and

we will look enhancing the same

Typical Main Battle Tank

CONTEXT

A main battle tank (MBT), also known as a

battle tank or universal tank, is a tank that fills the

heavy direct fire role of many modern armies.

Image Source: http://upload.wikimedia.org/wikipedia/commons/5/5b/US_Navy_050619-N-6811L-216_An_Australian_Army_Leopard_AS1_main_battle_tank_moves_up_the_beach_at_Sabina_Point_after_being_transported_ashore_by_a_U.S._Navy_Landing_Craft,_Air_Cushion_(LCAC).jpg

Slide 3

Features for the game

today

GETTING

STARTED

Arjan needs to

travel in shallow

water (upto 5 m) –

it can travel in

mud and sand.

Upgrade the main

firing gun from

90mm to 130mm

caliber

Improve the

weapon delivery

control system –

including gun data

computer, director

and radar

Introduce ceramic vehicle armourThis might impact the performance of the tank in durability, moving in

multiple terrains, withstanding the impact of firing (from the tank & on the

tank). This (and a combination of these) may be simulated in specialized test

Environments and may require prolonged monitoring.

Slide 4

Our Plan today

AGENDA

BACKLOG CREATION10 Minutes

ITERATION I10 Minutes

RETROSPECTIVE10 Minutes

DISCUSSIONS 30 Minutes

Slide 5

Let us start with the Tank Game!

Elicit Requirements

Create Backlog

Features > User Stories

EXERCISE

10 Minutes

Backlog Creation

Slide 6

Let us continue with the Tank Game!

Iteration 1 Planning

Execute Iteration

Iteration 1

EXERCISE

10 Minutes

Slide 7

Let continue with the Tank Game!

Iteration 1 Retrospective

EXERCISE

10 Minutes

Iteration 1 Retro

Slide 8

Let us look back

DISCUSSIONS

REQUIREMENTS

Did you get your

User Stories right?

GROOMING

Typically they

„emerge“ with lots

of discussions

ELICITATION

Was it straight forward?

BACKLOG

Did you include

NFRs in your

backlog?

We make things good. We make things big. We make things fast.

Slide 9

Let us look back on

what happened within

the Iteration

DISCUSSIONS

Plan

Build

Desig

nTest

Fra

me

Define Cycle

How were the

NFRs incorporated

in Iteration I?

How were these planned

to be verified?

Was the

Acceptance

smooth?

Acceptance

Slide 10

Nature of NFR testing

and defects pose a

challenge

CONTEXT

Complex set-up

Tools, test

organization and

execution is complex

Long Duration Tests

Tests take time with extended

monitoring

Need for expert

services

Specialized test

environment

Hard to reproduce

defects

Defect occurrance does

not follow a pattern

Design flaws

Defects may expose

architecture or design

flaws; Hard to fix

Slide 11

DEFINITIONNFRs can be

incorporated as individual

user stories.

Minor ones can be added

as „Acceptance Criteria“.

VERIFICATIONTest early – do what is possible.

Consider having parallel NFR test

iterations to the development

iterations where the previous

iterations software is subjected to

NFR tests.

Testing NFRs -

Recommendations

CONTEXT

FULL

VIEWFinal summary

testing at release

level.

Slide 12

Case in Point: Emergency call

Center product

EXAMPLE

An emergency call center application involving integration of multiple audio / VoIP

technologies. Has to comply with various safety and security standards.

Sprint 1

•NFR environment preparation

Sprint 2

•NFR Testing with Sprint 1 output

Sprint 3

•NFR Testing with Sprint 2 output

Defect Defect Defect Product Backlog

Release

level

Test ing

Case Source: http://www.slideshare.net/salem_iyer/scrum-gathering-indiaregional2013loadandperformancetestsinagilescrumframeworkv4

Slide 13

Crispin’s Agile Testing

Quadrants

AGILE TESTING

Image Source: http://img3.douban.com/view/page_note/large/public/p14837373-1.jpg

Slide 14

Crispin’s Agile Testing

Quadrants – applied to

our discussion

AGILE TESTING

Within Iteration Outside of Iteration

Slide 15

Look out for contexts where -

CONTEXTUAL

FITMENT

Complex

Set-upsDomain centric

Hardware Intensive

Safety Criticality

/ High Standards

compliance

Heterogenous

environment

Image Source: http://www.portlandwebworks.com/sites/default/files/stacey.png

Stacey‘s graph

QUESTIONS &

ANSWERS

Go ahead. Don‘t hesitate.

Slide 17

Let us stay in touch!

CONTACT

Twittertwitter.com/srv_subbu

LinkedInlinkedin.com/in/subrahmaniamsrv

E-MailSrv.subbu@yahoo.in

THANK

YOU

For your attention!