You cant control what you cant measure - Measuring requirements quality

SWQL Präsentationsvorlage V8.0

Software Quality Lab

Markus UnterauerConsultant and trainer

You can‘t control what you can‘t

measureMeasuring requirements quality

www.software-quality-lab.com | improve your quality


At the end of this session you know …Agenda

Why good requirements are important

What makes good requirements

How you can measure requirements quality

How to define metrics using G-Q-M

How you use metrics to improve

Slide 2



… and how reality often looks like

Why good requirements are

important

Slide 3



Requirements are the foundation of every SWRequirements quality

Slide 4

Source: fpm1979 via Flickr.com (https://www.flickr.com/photos/fpm1979/9660419677)



… and the source of all evilRequirements quality

Slide 5

Bad requirements

Unrealistic plans

Unplanned activities

Activities are

unpredictable

DelaysUnfulfilled

expectations

Poor quality

Misunderstandings

Time pressure

Overtime

Defects

So

urc

e: Ja

me

s W

. G

ren

nin

g, R

en

ais

sa

nce

So

ftw

are

Co

nsu

ltin

g C

om

pa

ny, B

osto

n 2

00

8



If we specify badly …Requirements quality

Slide 6

Source: Thomas R. Stegelmann via Flickr.com

https://www.flickr.com/photos/thomasrstegelmann/2731846156Source: Joel Kramer via Flickr.com

https://www.flickr.com/photos/75001512@N00/4518848983



… we often realize it (too) lateRequirements quality

Slide 7

55% aller Fehler entstehen in der

Anforderungs– und Entwurfsphase

Prozent aller

Entwicklungs-

fehler

Anforderungs-

und Entwurfs-

phase

Technische

Entwurfsphase

Konstruktion-–

und System-

testphase

Abnahmetest

und Betriebs-

phase

Entwicklungs-

phasen

60

50

40

30

20

10

Entwurfsfehler Logische Fehler Syntaxfehler

E

E

E

G

G

G

G

E = Eingebrachte FehlerG = Gefundene Fehler

Fehlerbeseitigungskosten (abgeleitet von Alberts 1985)

(Zitiert nach: IEEE Software, Jan. 1985, S. 83)

Most requirements

defects are found

in operation phase!

Percentage

of all

development

defects

Development

phase

Design defect Logical defect Syntax defect

Requirements

and conception

Technical design Construction and

system test

Acceptance and

operations

E = Entry point of defect

G = Defect found

55% of all defects occur in

requirements and conception phase



… what results in high follow up costsRequirements quality

Slide 8

Idea Specification Implementation Testing Operations

Effo

rtfo

r d

efe

ctco

rre

ctio

n

5minutes1hour

1 PD

3 PD

15 PD

Phase, in which a requirement defect is found

Effort for correcting a requirements defect

depending on when the defect is found

The later a

requirement defect is

found, the more

effort is needed for

correction



Status quo and causes

Slide 9



Requirements Engineering figuresRequirements quality

Productivity

75% of all companies do RE insufficiently

61% are unsatisfied with execution, 70% with documentation

50% of all success factors for projects lie in RE

Efficiency

16% of total project effort is used for RE

Quality

53% of all requirements are written in prose text

2-3% requirements creep per month

40-50% of total effort used for correcting RE related defects

58% of all testers see insufficient requirements as biggest challenge

Slide 10

[Eb

ert

], [W

ing

rove

et a

l.],

[Jo

ne

s],

[S

wis

sQ

Con

su

ltin

g A

G],

[S

tan

dis

h R

ep

ort

], ]Y

an

g e

t a

l.]



… and how to get them

We need better requirements!

Slide 11



Measurement as basis for all improvementsRequirements quality

Slide 12

Define goal and

path to reach itDiscover, where we are

Identify gap, to where

we should be

Find corrective measures

Go next step Goal reached



Proper metrics show the way to our goalHow to measure requirements

Slide 13

So

urc

e: D

ch

icvia

Flic

kr,

htt

ps:/

/ww

w.f

lickr.

co

m/p

ho

tos/a

ge

ncia

mod

els

/61

181

716

75



Boundaries show, when to actHow to measure requirements

Slide 14

Upper boundary

Lower boundary

Ideal value

Measured value

Time



Criteria for good requirementsRequirements quality

Reader acceptance criteria Synchronized

Valid and current

Realizable

Understandable

IEEE 830-1998 Rated

Unambiguous

Correct

Consistent

Testable

Traceable

Complete

Quality criteria based upon IREB® Certified Professional for Requirements Engineering

| Slide 15



Define proper metrics with G-Q-MHow to measure requirements

1. Goal What should be done for whom?

Which aspects do we want to change? What is the context?

e.g. „Improve the quality for all new projects“

2. Question Which questions guide us to the measurement goal?

e.g. „How understandable are our requirements?“

3. Metric How can we answer the questions using metrics?

e.g. „Average sentence length in words“

Slide 16

Define goals

Find

questions

Answer

questions



ExampleHow to measure requirements

Slide 17

Metric

Question

Goal We want to improve thequality of our specifications!

How unambiguous are ourrequirements?

Number ofweak phrases

Number ofsentences in

passive form

How complete are ourrequirements?

Number ofempty chapters



Question: How unambiguous are our requirements?Goal: Improve requirements quality

Percentage of weak phrases („could“, „should“, „rather“, „mostly“, …) within the top 20

Percentage of foreign words and abbreviations

Percentage of sentences with subordinate clauses

Number of universal quantifiers („all“, „never “, „none“, …)

Average sentence length (number of words)

Number of complex terms (terms with „-“)

Models per chapter

Percentage of concatenated sentences („and“, „but“, …)

Percentage of comparatives („faster“, „easier“, …)

Slide 18



Document metrics in a metric handbookHow to measure requirements

Slide 19

Metric Weak Phrases

Goal Specification must be unambiguous. Weak phrases point out unclear

requirements.

Level Sentence

Value Less misunderstandings

Less effort for discussions

Formula wp = nwp / s * 100

wp ….. weak phrases

nwp … number of weak phrases

s …… number of sentences

Method Automatic calculation using VBA macro

Datasource Word version of specification

Interval Before each sprint

Responsible Product Owner



Reporting and analysisHow to measure requirements

Observation Because of the high number of

concatenated sentences and fillerwords, the average sentence lengthis too long

Valuation Understandability: 82%

Unambiguity: 78%

Recommendations Split up long sentences

(„One requirement per sentence!“)

Eliminate filler words

Slide 20

Sentence length

in words

Filler words

within top 20

Weak phrases per

100 sentences

Tables with

> 30 rowsConcatenated

sentences per 100



Sample metrics with target rangeHow to measure requirements

Slide 21

Metric Target range

Sentence length (in words or lines) < 15 (ca. 1.5 lines)

Percentage of weak phrases< 0,25 per page

0 within Top 20

Percentage of sentences with

subordinate clauses< 15%

Number of tables with > 30 rows 0

Percentage of concatenated sentences < 5%



Efficiency in measurement means automationHow to measure requirements

Slide 22

Size:

- Pages: 5

- Words: 1066

- Sentences: 104

Complexity:

- Words per sentence: 13,25

- Tables with > 30 rows: 0

Weak phrases:

- „as possible“: 4 (every 26 sentences)

- „rarely“: 2 (every 52 sentences)

- „often“: 2 (every 52 sentences)

- „many“: 2 (every 52 sentences)

- „some“: 2 (every 52 sentences)

…



Tips for starting your requirements quality metric system

How to measure requirements

Define metrics top down, not bottom up

Goal > Question > metric

Not: „Hey, look how many things we can measure!“

Concentrate on 5 - 10 metrics, e.g. these:

Sentence length (in words or lines)

Percentage of weak phrases

Percentage of sentences with subordinate clauses

Number of tables with > 30 rows

Percentage of concatenated sentences

Define lower and upper boundary together with your team … or measure without boundaries for some iterations and define boundaries then

Start with automatically measurable measures

Define, what to do with the metrics

Slide 23

Image: htt

p:/

/pix

abay.c

om

/en/b

ulb

-lig

ht-

lam

p-e

lectr

ic-1

60207/



SummaryKey learnings from this session

Poor requirements lead to bad software

Measuring requirements quality using metrics is essential Objectivly judge quality

Derive actions and evaluate effectiveness

Use GQM method for metric definition G – Define goals

Q – Ask questions

M – Define metrics to answer the questions

Automate measurement

Slide 24


Your Partner in Software Quality and Testing

Software Quality Lab GmbH

www.software-quality-lab.com

Consulting | Service | Academy | Tool Expertise

Markus [email protected]

You cant control what you cant measure - Measuring requirements quality

Technology

Transcript of You cant control what you cant measure - Measuring requirements quality