1

Using Formal Methods inIndustrial Software Development

European SEPG ConferenceLondon, June 2005

Robert van Lieshout, Quality ManagerRobert.vanLieshout@imtech.nl

2

“Quality through Commitment and Creativity”

3

Engineering and MathematicsEvery branch of Engineering uses Mathematics for Specification, Design and Verification

Mechanical Engineering => Differential EquationsStructural Engineering => Finite Element AnalysisCircuit Engineering => Boolean Algebra, etc

Except Software EngineeringOriginally a specialization within Mathematics!

Most Software is specified and designed without using mathematicsSoftware specifications and designs cannot be verified before

implementationSoftware testing must find specification, design and

implementation errors

4

Presentation Outline

Formal Methods – then and nowI-Mathic – an overviewApplying I-Mathic – some resultsBenefits and DrawbacksCurrent status

5

Formal Methods

Then and now

6

Formal Methods – Then ...

Formal Methods have promised much and delivered little:The solution is often more complicated than the problem

Formal specifications use difficult notations and require extensive mathematical backgroundCritical Stakeholders - Business Analysts, Domain Experts and Customers - cannot understand the formal specifications

Critical Stakeholders excluded from the process

7

Formal Methods – Now

Growing interest in Formal MethodsSee: http://www.fmeurope.org/Several methods with different objectivesProof of selected properties, rather than full correctness proof

Tool support and computing power have:Reduced laboriousnessImproved user-friendliness (sometimes)Made it less time consuming

8

The I-Mathic Formal Method

An overview

9

I-Mathic Principles

Design PrincipleDevelopers can and should strive to produce software that is nearly error free when entering testing

Testing PrincipleThe purpose of testing is quality measurement and not an attempt to “test in” quality

10

I-Mathic – Origins

Used a small part of Cleanroom Sw. Eng.Sequence Enumeration

Developed tool supportExcel template with VB automationScripts for visualisationScripts for code & test generation

Linked it with CSPCSP: Communicating Sequential ProcessesTools to check the CSP model

11

I-Mathic – Overview

12

Applying I-Mathic

Some results

13

Run outRun in

PP PP PP … PP PP

Transport system containing PCB’s

1..20 Pick & Place Robots

Example Project – Assembléon AX

14

AX Kernel

TCPPC SVS UTLMonitor

GUI intf(29 states)

Process Ctrl

‘Glue’

GPETERM GPEGPEGPE

Module intf(10 states)

Glue intf(11 states)

GPE intf(17 states)

Example Project – AX Kernel

15

Example Sequence Enumeration (fragment)0:<> IDLE

1 Cl.rqStartProduction Mod.rqStartProduction 1 GPE.Req12 Cl.rqPauseProduction Illegal - GPE.Req23 Cl.rqRecover Illegal - GPE.Req34 Srv.ntErrorOccured Illegal - GPE.Req15 Srv.ntProductionStarted Illegal - GPE.Req26 Srv.ntProductionPaused Illegal - GPE.Req37 Mod.ntErrorOccured Illegal - MOD.Req18 Mod.ntProductionStarted Illegal - MOD.Req29 Mod.ntProductionPaused Illegal - MOD.Req3

1:<Cl.rqStartProduction> STARTING_MOD10 Cl.rqStartProduction Illegal - D011 Cl.rqPauseProduction Illegal - D012 Cl.rqRecover Illegal - D013 Srv.ntErrorOccured Illegal - GPE.Req114 Srv.ntProductionStarted Illegal - GPE.Req215 Srv.ntProductionPaused Illegal - GPE.Req316 Mod.ntErrorOccured Cl.ntErrorOccured 16 Module in ERROR MOD.Req117 Mod.ntProductionStarted Srv.rqStartProduction 17 Module in RUN state MOD.Req218 Mod.ntProductionPaused Illegal - MOD.Req3

The Component Function is completeMaps every possible input sequence to response

The Component is the right systemEvery transition rule justified, full requirements tracingDerived requirements fill the gaps

16

Results – Model CheckingModel checker explores all state combinations of the CSP

model ensuring that:

Model is deterministicModel implements interface according to specificationThere are no deadlocksFinite queue size is sufficientQueues are never full (processes behave freely)

17

Results – Project Performance

122131143Productivity (eLocs/man day)

38,581,5112Effort (man days)

46901071516050Code Size (eLocs)

39012403000Transitions

2038194States

CBA

0.00.30.2Post Release Defects per 1000 eLoc

033Post Release

000During Acceptance

575During Internal verification

CBADefects

Size and Effort

18

Example – Quick ScanUsed to model part of an existing system

System Characteristics:Data driven multi-media application Multi-threaded components“Mature software”

Approach:Reverse-engineered models for a few scenario’s with help from domain expertRan model checks

19

Example – Quick ScanScope:

2 interfaces & 1 implementation modelledInterface 1: 5 states, 12 transitionsInterface 2: 10 states, 39 transitionsImplementation: 6 states, 44 transitions

Result:Potential deadlock detectedSeveral undocumented requirements revealedTotal effort spent: <12 man days

Including customer effort & writing report

20

Applying I-Mathic

Benefits and Drawbacks

21

Benefits

Encourages interaction with stakeholdersSufficiently understandableConsistent and complete requirements

Few defectsDeadlock free; No race conditionsSimple defects, easy to solve

High productivityPartly due to code generationPartly due to reduced test effort

22

Drawbacks

Limited integration with other methods and tools

I-Mathic is a very different approach

Requires abstract thinking and disciplinewhich not all software engineers have!

23

How does it scale?

Current data is on relatively small projectsExpected to scale well, considering the integration effort and defects

Maybe next year we’ll know?

24

I-Mathic

Current Status

25

Current status

Mandated for use in in-house projectsQuick ScanSupport for multi-threaded componentsRegular additions to toolset

E.g. context sensitive menu’sOngoing development:

Research together with Dutch universities

26

References

Cleanroom Software EngineeringMills, H.; Dyer, M.; & Linger, R.IEEE Software, September 1987

Communicating Sequential ProcessesHoare, C.A.R.Prentice Hall, April 1985ISBN: 0131532715

27

Thank you

Any questions?Robert van Lieshout

Robert.vanLieshout@imtech.nlwww.imtechict.nl