Gutes RE, schlechtes RE

Best PracticeRequirements Engineering

i4DS Centre for Requirements Engineering

Gutes RE, schlechtes RE

04.01.2017i4Ds | Centre for Requirements Engineering 2

References

IREB Foundation Level Syllabus: https://www.ireb.org/en/cpre/foundation/

S. Fricker, R. Grau, A. Zwingli (2014): “Requirements Engineering: Best Practice” in S. Fricker. C. Thümmler, A. Gavras: Requirements Engineering for Digital Health. Springer. http://www.diva-portal.org/smash/get/diva2:834026/FULLTEXT01.pdf

SwissQ (2016): Software Development 2016 Trends& Benchmarks Report Schweiz. http://report.swissq.it/de/

https://www.ireb.org/en/cpre/foundation/

http://www.diva-portal.org/smash/get/diva2:834026/FULLTEXT01.pdf

http://report.swissq.it/de/

Samuel Fricker

Professor for Requirements Engineering, FHNW

Assistant Professor for Software Engineering, BTH

Doctor in Informatics (Requirements Engineering) Uni Zürich

ABB, Fuchs-Informatik, Startups…


About Us: i4Ds Centre for Requirements Engineering

Farnaz FotrousiPhD Student

Melanie StadePhD Student

Prof. Dr. Norbert SeyffProvessor

Prof. Dr. Samuel FrickerProfessor


Centre for Requirements Engineering

Research and Innovation for Aligning Technology with Society’s Needs

i4DS Centre for Requirements Engineering


What is ‘Best Practice’?

Google: commercial or professional procedures that are accepted or prescribed

as being correct or most effective.


IREB Standard Practice

Elicitation: Creativity, Surveys, Document Analysis, and Observation

Analysis: System Interfaces and Context

Documentation: Documents, Shall-Templates, UML, SA, Goal Diagrams

Checking: Checklists, Review Techniques, Handling of Conflicts

Management: Attributes, Prioritization, Traceability, Baselining, Changes

Tools and Tooling


Requirements Engineering



Idea: Best Practiceis Common Practice


Survey Research in 2012


Survey Research

State-of-art

Panel of Experts

Questionnaire

Statistical Analysis


419 Projects

< 10 23 5% Banking, Finance 98 23% Research 32 8%10-49 39 9% Automotive, Transport 54 13% Product, Platform 87 21%

50-249 42 10% Software, IT 51 12% Bespoke 237 57%250-4499 144 34% Government, Military 40 10% Tender 42 10%

>= 4500 168 40% Healthcare, Medical 35 8% Other 21 5%n/a 3 1% Insurance 31 7%

Telecommunications 31 7% Prototyping 23 5%Europe 368 88% Manufacturing, Supply 22 5% Evolutionary 39 9%

Switzerland 248 59% Other 57 14% Incremental 113 27%Germany 69 16% Hybrid 104 25%

Other Europe 51 12% Information System 260 62% Sequential 135 32%Americas 26 6% Software-Intensive 62 15% Other 5 1%

Asia-Pacific 25 6% Engineering 28 7%Other 69 16% < 4 54 13%

4-9 107 26%Completely New 149 36% 10-19 89 21%

New Features or Use 107 26% 20-49 84 20%Changed Solution 79 19% >= 50 82 20%

Enhancement 84 20% n/a 3 1%

< 4.5 27 6%4.5-9 97 23%9-18 119 28%

>= 18 165 39%n/a 11 3%

Duration (calendar months)***

ProductCategory

Process (Lifecycle Model)

Size (number of staff)***

Location

ProjectIndustry (application domain)

Type

Innovation

CompanySize (number of employees)***


Requirements Engineering

Product Planning Requirements Inquiry Requirements Management

Goals

Requirements

Design(Constraints)

Tests

code

Manuals

Inputs

Stakeholders

Project Team

Analyst

ProductManager

Software Project


The Big Picture: Common Practice

Total 405 97% Total + 414 99% Total 384 92% Total 407 97% Total + 404 96%Reqs. Prioritizing 252 60% Workshops 328 78% Informal Modeling + 210 50% Functional 343 82% Natural Language 374 89%Release Planing 209 50% Feedback ± 183 44% Prototyping + 169 40% Scenarios ± 263 63% Use Cases 248 59%

Requirements Triage 206 49% Analysis ± 161 38% OOA ± 166 40% Quality 240 57% Informal Text 219 52%Business Case 202 48% Design 149 36% Quality Checks 107 26% User Interfaces 238 57% User Stories 111 26%Roadmapping 174 42% Creativity 142 34% SA 51 12% Processes ± 183 44% Shall Templates 94 22%

Vision ± 165 39% System Archeology 292 70% DDD 34 8% Rules 173 41% Other 37 9%Other 1 0% Requirements Reuse 270 64% Other 36 9% Software Interfaces 157 37% UML Diagrams 245 58%

Copy/Paste 159 38% Structure 140 33% Use Case Diagrams 188 45%Total 382 91% Delta Specification 121 29% Total + 391 93% Glossary ± 132 32% Activity Diagrams 128 31%

Reqs. Prioritizing 252 60% Standard Reqs. 81 19% Inspection + 266 63% Behavior 95 23% Class Diagrams 114 27%Handshaking 209 50% Variability Analysis 42 10% Walk-Through + 175 42% Agents 71 17% Sequence Diagrams 89 21%

Conflict Management 167 40% Modeling-based 3 1% Peer/Advisor Review 161 38% Formal Properties 24 6% State Machines 54 13%Strategy Alignment 125 30% Interviews + 265 63% Prototype Review 143 34% Other 26 6% Other 2 0%

Power Analysis 76 18% Document Analysis 211 50% Checklist ± 89 21% Graphical Processes 208 50%Win-Win Negotiation 45 11% Creativity 183 44% Simulation 33 8% Total 405 97% Activity Diagrams 128 31%

Variant Analysis 31 7% Workshops 142 34% Automated Checking ± 30 7% Document 265 63% DFD 111 26%Negotiation Analysis 29 7% Idea Castings 43 10% Other 4 1% Spreadsheet 149 36% BPMN; BPML 37 9%

Idea Databases 38 9% Database 146 35% Other 9 2%Total 341 81% Introspection 118 28% Modeling Tool 135 32% SA Diagrams 177 42%

Change Management 243 58% Observation + 87 21% Drawing Tool 61 15% DFD 111 26%Baselining 196 47% Surveys 50 12% Other 4 1% ERD + 94 22%

Traceability 167 40% Data Mining - 25 6% STD 62 15%Progress Tracking 106 25% Other 12 3% User Screens 151 36%

Report Generation 60 14% Informal Drawings 139 33%Process Measurement 55 13% Tables 67 16%

Other 3 1% Other 19 5%

Multiple answers were possible, + and -: significant changes compared with earlier surveys3, 4, ±: no change compared with earlier surveys according to Holm's step-down methodlight color: sub-categories, "Other": answers with less than 5% frequency

Requirements Management

ManagementProduct Planning Elicitation Requirement TypesAnalysis

Storage

Checking

Inquiry Specification

Stakeholder Negotiation

Notations


Common Practice: Elicitation

Almost every project elicited requirements.

Projects tended to–use stakeholder collaboration and existing

systems or specifications…–…more often than creativity and knowledge

of project members.

Total + 414 99%Workshops 328 78%Feedback ± 183 44%

Analysis ± 161 38%Design 149 36%

Creativity 142 34%System Archeology 292 70%

Requirements Reuse 270 64%Copy/Paste 159 38%

Delta Specification 121 29%Standard Reqs. 81 19%

Variability Analysis 42 10%Modeling-based 3 1%

Interviews + 265 63%Document Analysis 211 50%

Creativity 183 44%Workshops 142 34%

Idea Castings 43 10%Idea Databases 38 9%

Introspection 118 28%Observation + 87 21%

Surveys 50 12%Data Mining - 25 6%

Other 12 3%

Elicitation

i4Ds | Centre for Requirements Engineering

Common Practice: Notations

Almost every project documented requirements–often with natural language.

Projects preferred pragmatic notation use over method compliance.

Total + 404 96%Natural Language 374 89%

Use Cases 248 59%Informal Text 219 52%

User Stories 111 26%Shall Templates 94 22%

Other 37 9%UML Diagrams 245 58%

Use Case Diagrams 188 45%Activity Diagrams 128 31%

Class Diagrams 114 27%Sequence Diagrams 89 21%

State Machines 54 13%Other 2 0%

Graphical Processes 208 50%Activity Diagrams 128 31%

DFD 111 26%BPMN; BPML 37 9%

Other 9 2%SA Diagrams 177 42%

DFD 111 26%ERD + 94 22%STD 62 15%

User Screens 151 36%Informal Drawings 139 33%

Tables 67 16%Other 19 5%

Notations

Total 384 92%Informal Modeling + 210 50%

Prototyping + 169 40%OOA ± 166 40%

Quality Checks 107 26%SA 51 12%

DDD 34 8%Other 36 9%

Analysis


Common Practice: Checking

Almost every project checked requirements,–often by formal inspection.

Projects typically–preferred manual requirements checking…–…over simulation and automated checking

Total + 391 93%Inspection + 266 63%

Walk-Through + 175 42%Peer/Advisor Review 161 38%

Prototype Review 143 34%Checklist ± 89 21%

Simulation 33 8%Automated Checking ± 30 7%

Other 4 1%

Checking


Common Practice: Tools

Almost every project stored requirements.

Projects tended to–prefer writing techniques (documents)…–…over list-oriented techniques

(spreadsheets, databases).

Uncommon was storage of requirements in drawing tools, wikis, and cards.

Total 405 97%Document 265 63%

Spreadsheet 149 36%Database 146 35%

Modeling Tool 135 32%Drawing Tool 61 15%

Other 4 1%

Storage



Idea: Best PracticeCorrelates with RE Success


Success Criteria

Total 419 100%Productivity 228 54%

Effectiveness 156 37%Compliance 143 34%Satisfaction 137 33%

Flexibility 86 21%Safety 73 17%

Environment 7 2%Other 8 2%

Software Product GoalsTotal 419 100%

Shared Understanding 214 51%Specification Quality 197 47%

Clear Scope 160 38%Efficiency 155 37%

User Satisfaction 145 35%Timeliness 139 33%

Fit of Solution 94 22%Estimation Reliability 65 16%Architecture Quality 58 14%

Cost/Benefit Analysis 26 6%Other 4 1%

RE Constraints


Success and Failure

Too Little 181 43%Just Enough 229 55%

Too Much 9 2%

Rather Yes 385 92%Yes 182 43%

Likely 203 48%Rather No 34 8%

No 18 4%Unlikely 16 4%

SuccessRequirements Engineering Amount

Product Goal AchievementANDOR

221 successes189 failures


The Big Picture: Success-Correlating Practices

218 99% 220 100% 207 94% 219 99% 219 99%178 94% 185 98% 168 89% 179 95% 176 93%

135 61% 189 86% 101 46% 192 87% 199 90%111 59% 133 70% 102 54% 142 75% 168 89%

126 57% 114 52% 99 45% 160 72% 142 64%71 38% 66 35% 65 34% 100 53% 104 55%

118 53% 95 43% 99 45% 141 64% 113 51%84 44% 56 30% 66 35% 97 51% 102 54%

114 52% 89 40% 66 30% 138 62% 68 31%91 48% 62 33% 37 20% 95 50% 43 23%

98 44% 88 40% 32 14% 104 47% 46 21%72 38% 53 28% 18 10% 73 39% 44 23%

97 44% 152 69% 19 9% 98 44% 22 10%65 34% 133 70% 14 7% 72 38% 14 7%1 0% 148 67% 24 11% 94 43% 138 62%0 0% 115 61% 11 6% 61 32% 102 54%

78 35% 82 37% 106 48%76 40% 49 26% 79 42%

208 94% 61 28% 214 97% 81 37% 71 32%165 87% 58 31% 169 89% 56 30% 54 29%

135 61% 56 25% 148 67% 52 24% 68 31%111 59% 23 12% 112 59% 42 22% 44 23%

124 56% 29 13% 97 44% 46 21% 48 22%79 42% 13 7% 61 32% 25 13% 39 21%

96 43% 3 1% 91 41% 20 9% 31 14%66 35% 0 0% 50 26% 4 2% 21 11%

78 35% 147 67% 91 41% 15 7% 2 1%46 24% 113 60% 81 43% 12 6% 0 0%

52 24% 111 50% 51 23% 114 52%22 12% 71 38% 37 20% 88 47%

30 14% 88 40% 20 9% 217 98% 71 32%13 7% 53 28% 12 6% 179 95% 54 29%

20 9% 28 13% 18 8% 148 67% 68 31%10 5% 15 8% 11 6% 110 58% 40 21%

21 10% 22 10% 3 1% 84 38% 23 10%8 4% 16 8% 1 1% 64 34% 13 7%

104 47% 80 36% 4 2%101 53% 52 28% 3 2%

194 88% 57 26% 77 35% 101 46%139 74% 57 30% 65 34% 72 38%

142 64% 44 20% 39 18% 68 31%93 49% 41 22% 21 11% 40 21%

112 51% 31 14% 1 0% 56 25%79 42% 18 10% 3 2% 35 19%

104 47% 15 7% 33 15%59 31% 10 5% 28 15%

67 30% 7 3% 84 38%36 19% 3 2% 63 33%

37 17% 76 34%22 12% 61 32%

33 15% 36 16%21 11% 29 15%1 0% 15 7%2 1% 4 2%

Total

Requirements Management

Stakeholder Negotiation

Product Planning Elicitation Analysis

Checking

Requirement Types

Other

Simulation

Automated Checking

Other

Functional*

Scenarios*

Quality

User Interfaces

Processes

Rules

Software InterfacesOther

Inspection

Peer/Advisor Review

Prototype Review*

Walk-Through

Tables

Other

Total Total Total* Total*

Storage

Natural Language

UML Diagrams

Graphical Processes

SA Diagrams

User Screens

Informal Drawings

Document

Spreadsheet

Modeling Tool

Database

Drawing Tool

Other

Glossary

Structure

Behavior

Agents

Formal Properties*

Informal Modeling

OOA

Prototyping

Quality Checks

SA

DDD

Workshops*

Total*

Feedback*

Design*

Analysis

Creativity

Other

DFD

ERD

STD

System Archeology

Requirements Reuse

Interviews

Creativity

Document Analysis

Introspection

Sequence Diagrams

State Machines

Other

Activity Diagrams

DFD

BPMN; BPML

Idea Castings

Idea Databases

Variability Analysis

Modeling-based

Checklist

Observation

Surveys

Data Mining

User Stories

Shall Templates

Other

Use Case Diagrams

Activity Diagrams

Class Diagrams

Copy/Paste

Delta Specification

Standard Reqs.*

Win-Win Negotiation

Variant Analysis

Negotiation Analysis

Change Management*

Baselining

Traceability*

Progress Tracking*

Report Generation

Process Measurement

Other

Workshops

Total*

Other

Notations

Reqs. Prioritizing

Total

Reqs. Prioritizing

Handshaking*

Conflict Management

Strategy Alignment

Power Analysis*

Total

Business Case*

Requirements Triage

Release Planing

Roadmapping

Vision

Other

Use Cases

Informal Text


3 practicescorrelated

with RE Success

Business Case

A business case predicts financial results and other business consequences.

126 57%71 38%

Business Case*

Source: M. Schmidt: The Business Case Guide.

Scenarios

Scenarios document exemplary sequences (stories) of system usage.

160 72%100 53%

Scenarios*

Buy Ticket:1) Traveller enters destination2) Ticket machine shows ticket options3) Traveller selects option4) Ticket machine shows price5) Traveller enters money6) Ticket machine issues ticketAlternatives…Exceptions….

Workshops

Workshops create an efficient, controlled, and dynamic setting for quickly eliciting, prioritizing, and agreeing on requirements.

189 86%133 70%

Workshops*


Use of Success-Correlating Practices

92, 71%

38, 29%

Did Use Business Case, Workshops, and Scenarios

12, 31%

27, 69%

Did NOT use BC, WS, or SC



RefsQ 2015: “I heard it first at RefsQ”

Idea: Best Practice should beModern or Mature Practice


The Winner was…

< 10% 2-Taken Up 3-Mature 4-Declining

ADO

PTIO

N

Maturity


Practice Hype Cycle

Please Vote from a Practice Perspective: What Topics Go in Which Phase?

1-New


Practice Hype Cycle

Most frequent answers from GI-RE meeting participants (Nov 24, 2016)

0

5

10

15

20

25

New Growing Mature Declining

Automated RE

Agile RE

User Feedback

Workshops

User Stories

SA

Goals

Interviews

Use Cases

UML

Workshops

SA

UserStories

Automated RE


Modern Practices: A Swiss View

Consistent, longitudinal research of practice use:–2013–2014–2015–2016


Modern Practices: A Swiss View

Consistent, longitudinal research of practice use:–2013–2014–2015–2016

I aggregated by looking at–average practice use

over all four years–difference between

average 2013/2014 andaverage 2015/2016


Extent and Change of Practice: Elicitation

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

-50% -40% -30% -20% -10% 0% 10% 20% 30% 40% 50%

Exte

nt o

f Use

Growth

Interview

Introspection(Use of Experience)

ArchaeologyWorkshops

ReuseOn-site Customer

Observation

Surveys

Creativity


Extent and Change of Practice: Specification

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

-50% -40% -30% -20% -10% 0% 10% 20% 30% 40% 50%

Exte

nt o

f Use

Growth

Informal Text

User StoriesUser ScreensUse Cases

Informal DrawingsUse Case Diagrams

BPMNActivity Diagrams

DFD Class DiagramsSequence Diagrams


Extent and Change of Practice: Checking

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

-50% -40% -30% -20% -10% 0% 10% 20% 30% 40% 50%

Exte

nt o

f Use

Growth

Walkthrough

Reviews

Checklists Peer/Advisory ReviewSimulation

Prototype Review

Test Definition


Extent and Change of Practice: Tools

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

-50% -40% -30% -20% -10% 0% 10% 20% 30% 40% 50%

Exte

nt o

f Use

Growth

Documents

JiraPaper and Pen

EnterpriseArchitect

VisioConfluence

Quality Center

TFS

Balsamiq

DoorsPolarion

i4DS Centre for Requirements EngineeringProf. Dr. Samuel Fricker


Summary and Conclusions

Best Practice may be Standard Practice, Common Practice, Success-Correlating Practice, Modern Practice

Standard: requirements inquiry and management

Common: use of natural language, functional requirements, workshops

Success-correlating: business case, scenarios, and workshops

Modern practice: introspection, user screens, user stories, pen and paper, jira, confluence, Balsamiq

…how smart is practice, and what should we do about it?


How Smart is RE Practice?

Total: 67 practices

Natural Language 374 89%Functional 343 82%Workshops 328 78%System Archeology 292 70%Requirements Reuse 270 64%Inspection + 266 63%Interviews + 265 63%Document 265 63%Scenarios ± 263 63%Reqs. Prioritizing 252 60%UML Diagrams 245 58%Change Management 243 58%Quality 240 57%User Interfaces 238 57%Document Analysis 211 50%Informal Modeling + 210 50%Release Planing 209 50%Handshaking 209 50%Graphical Processes 208 50%Requirements Triage 206 49%Business Case 202 48%

3

9

21


Shall We Teach Requirements Engineering?

Successful projects used a wider variety of RE techniques and defined more requirement types than unsuccessful projects.

0

5

10

15

20

25

30

35

Avg. Number of RE Techniques

Successful RE Other RE

+25%

0

1

2

3

4

5

6

Number of Requirement Types

Successful RE Other RE

+26%

Gutes RE, schlechtes RE

Documents

Transcript of Gutes RE, schlechtes RE