Software Engineering with Formal methods

8/12/2019 Software Engineering with Formal methods

1/38

Software Engineering with Formal Methods :

The Development of a

Storm Surge Barrier Control System

Seven Myths of Formal Methods Revisited

Jan Tretmans

University of Twente

Klaas Wijbrans, Michel Chaudron

CMG Public Sector B.V.

With support from:

Franc Buve, Eric Burgers, Wouter Geurts,

Rijn Buve, Sjaak de Graaf, Hedde van de Lugt,

Peter Bosman, Peter van de Heuvel,

Robin Rijkers, Pim Kars, Ed Brinksma,

Wil Janssen, Theo Ruys, Job Zwiers

1


2/38

Overview

The storm surge barrier and BOS

Approach to Software Development

Experiences with the use of formal methods:

Seven Myths of Formal Methods revisited

(Seven Myths of Formal Methods,

Anthony Hall, IEEE Software 6(9), 1990)

2


3/38


4/38

R

The Hagu

Hoek van Holland


5/38

StormStormsurge barriersurge barrier

Rotterdam

360m

sea


6/38


7/38

The stormThe stormsurge barriersurge barrier

Kugelgelenk

Sand/Beton

Kugelgelenk

Sand/Beton


8/38

AsAsreliablereliableas aas a dykedyke

Risk of failure dike: 1 in 10.000 years

Extreme high water: 1 in 10 years

Risk of failure barrier: 1 in 1.000 closure


9/38

WhyWhya BOS?a BOS?

Risk of failure in decision: 1 in 100.00

Failure probability human: 1 in 1.000 -

Decision process must be done automatica

=> BOS


10/38

WhatWhatdoes BOS do?does BOS do?

Decide when to close the barrier

Decide when to open the barrier

Control the barrier in the Nieuwe Waterweg

Control the barrier in the Hartelkanaal Decide when maintenance is allowed

Decide if a test closure is allowed and perfo


11/38

HowHowdoesdoesit workit work??

Acquire data

Predict water levels

Decide and control

Archive


12/38

HowHowwaswasititbuilt?built?

Functional requirements drawn by RWS

Required reliability 1 in 100.000

Technical design and software by CMG (fix

25 person-years, 3 years from start to end 450.000 lines of code


13/38

HighHighreliabilityreliability

Hardware

Fault-tolerant Stratus Continuum computer

Multiple communication lines and a private satel

Software

Reliability 1 in 100.000 cannot be shown by testi

take 2000 year!

Certification according to ISO 9001

Standard for safety-critical systems: IEC 1508


14/38

ResultResult

Delivered october 1997 within budget and o

Since then it has rightly been in alert twice

On October 3, 1998 the first test closure


15/38

Standard IEC 61508

Recommendation for implementingsafety critical software

No measure of reliability or safety for software

Based on Best Practices

Recommends and forbids particular development

techniques based on Safety Integrity Level SIL:

Pfail 102 103 104 105

SIL 1 2 3 4

BOS

Highly Recommended for SIL 4:

inspections

independent testing

.....

formal methods

3


16/38

BOS Approach

Risk based / failure analysis

Well-defined project life-cycle,strict configuration management

Use of best, feasible practices

No guarantee of correctness, but increaseof confidence by combination of validationtechniques: Validation & Verification Plan

V & V plan:

reviews and inspections

coding standards and static checking

coding assertions

developer testing (white box) independent module/integration/system

testing with coverage measurement

formal specifications

model checking

simulation

4


17/38

Formal Methods in BOS

Starting Points

Goal:To increase the reliability and correctness

of the BOS software

Cooperation CMG University of Twente

Integration in development trajectory

not a parallel, academic exercise

Fixed time fixed price project

Non-formal (Dutch + Hatley & Pirbhai)

specification given

5


18/38

Formal Methods in BOS

How to Start ?

where to apply ?

which formal techniques ?

in which phases of development ?

how to learn FM ?

how to manage FM ?

how to combine with other

(non-formal) SE techniques ?

expected costs and benefits ?

6


19/38

Where to apply FM

GUI modelprediction

interfaces

specification

formal specification/design

argumentation

formal derivation of code

level of formality

components

design

implementation

testing

phases of development

proof

7


20/38

Seven Myths of Formal Methods

Are the seven myths of formal methods

really myths for the BOS project ?

( From: Seven Myths of Formal Methods,Anthony Hall, IEEE Software 6(9), 1990 )

1. FM guarantee correctness

2. FM are about program proving

3. FM are only for safety-critical systems

4. FM require highly trained mathematicians

5. FM increase development costs

6. FM are unacceptable to users

7. FM are not used on real software

8


21/38

Myth 1

FM guarantee correctness

BOS: no guarantee of correctness

increased confidence in correctnessthrough increased precision andearly detection of defects

9


22/38

No guarantee of correctness:

No single method can guarantee correctness

FM not applied with all their rigour:

System too large for manual proof

No usable tools for automatic proof Dont be too formal during

first time use of FM

Not all aspects of behaviour in single formalism

Not all system components formally specified

Difference formal model reality

Specification not formal

Implementation and testing

based on formal specifications

but no formal derivation or proof

10


23/38

Development Trajectory

specification

Technical

TDDesign

Testing

functional

FS

Implementation

11


24/38

Myth 2

FM are about program proving

BOS: no program proved, but

formal description of design model checking of some protocols

Result:

process of formally specifyingleads to early detection of errors

precise, unambiguous, complete

understanding, argumentation and simulation

precise basis for implementation and testing

Early phases most critical;implementation / testing no big problem

16


25/38

Myth 3

FM are only for safety-critical systems

BOS: safety-critical and FM used

Other systems:

almost all systems critical

safety, economic, company image

BOS experience:better quality with (almost) same costs

lot of costs in learning

17


26/38

Myth 4

FM require highly trained mathematicians

BOS: software engineers can learn FM

FM based on relatively simple mathematics

No need to learn all mathematical backgroundfor using FM

No complete proofsModel checking by few persons

No mathematicians, please:

If they have a model they can calculate

Problem: to get the valid model

they aim at elegant solutionsnot at practical ones

Not mathematics of FM is difficult

but learning effective usage in SE

18


27/38

Learning FM

learning formal languages not difficult

learning formal method difficult

difficulties:

how to make models / level of abstraction

what to formalize and what not how to manage FM

how to use the tools effectively

attitude and mentality change

developing specification styles / standards

developing coding techniques

developing reviewing techniques for FM

embedding in SE process

integration with existing methods

19


28/38

Learning FM

courses only to learn the formal language

university knowledge restricted to formal

language knowledge

learning usage on the job

potential conflict: learning project progress

guru (hero) necessary

potential conflict: guru task gurus task

breaking point in thinking about FM:before break: FM are a burden

after break: real benefits

how did we ever manage without FM?

not everybody likes FM

20


29/38

Myth 5

FM increase development costs

BOS: probably yes, a little bit

But:

No real comparison: no non-formal BOS

Much of costs in learning first time costs

More important:

shift in costs

from implementation and testing

towards specification and design

21


30/38

Shift in Efforts

Design phase more important and much longer

Gain in implementation and testing

Design more detailed

Earlier detection of problems / defects

first think and then build

In principle, independent from FM

but without FM easier to escape from it

Difficult to measure / manage progressduring design

Planning of implementation and testing

very precise

22


31/38

Myth 6

FM are unacceptable to users

BOS: user ( RWS ) was satisfied

informal explanations of formal specifications

are necessary

simulation of formal models was very useful

to show defects ( SPIN with MSC )

23


32/38

Myth 7

FM are not used on real software

BOS: used successfully

And with a next, BOS-like system,

CMG will use FM and try to increase the level of formality

Dad, when you were young, were there

really people who developed their software

without formal methods ?

How did they manage ?

24


33/38

Benefits & Costs

+ better software quality

+ more preciseness

+ problems and defects earlier detected

+ better basis for testing

+ no major problems found during testing

+ better planning of implementation

and testing phase

+ efficient reviewing of code based on

formal design with uniform style

+ estimated better maintainability

(if FM specs kept up to date)

25


34/38

Benefits & Costs

design phase longer;

implementation and testing phase shorter

increase in professionalization:

good projects get better; bad projects get worse

likely longer overall development,mainly due to learning

learning to make effective use of FM not easy

integration with current SE practice is weak

tool support is insufficient for large systems

26


35/38

Comparison with Seven Myths

Observations of Anthony Hall and BOS mostly agree

Minor differences of BOS w.r.t. Hall:

distinction specification design

distinction specification model

FM not ideal for conceptual modelling

and high-level structuring

importance of tools:

learning

stimulating

check of specifications

model checking

but insufficient:

- functionality

- size which can be handled

- integration with SE tools

27


36/38

Conclusion FM Usage

FM do increase quality

FM are usable in industrial context

FM require some learning and adaptation,

not completely off-the-shelf

28


37/38

Conclusion FM Research

Not so much need for new FM theories

but making existing FM theories

better applicable

Integration of data and process formalisms

Better tools needed w.r.t. to size of models

Integration within SE practice

Bottle-neck in early development phases

Proving code correct w.r.t. formal specification

is not important

Getting the formal specification is important

29


38/38

More about the Storm Surge Barrier:

http://www.minvenw.nl/rws/dzh/svk/engels/

Software Engineering with Formal methods

Documents

Transcript of Software Engineering with Formal methods