Holger Hermanns Formal Methods for Software Engineering Part III: Applications of Formal Methods...

Holger Hermanns

Formal Methods for Software Engineering

Part III:Applications of Formal

Methods

Lecture 10:Applications?

2 Holger Hermanns Formal Methods for Software Engineeiring - Lecture 10

This Part

We apply the concepts, methods and tools you learnt to love in contexts that are relatively close to what the people out there are facing.

This week I show you what they are facing,

and I round off the entire lecture series.

Assumptions for today:

Nothing particular.


Contents of this lecture

A real application.

Testing based on formal methods.

Another real application.

Model construction and model checking beyond what you have seen in this entire set of lectures.

A third, very real application.

Wrap up.


What’s this?


Nieuwe Waterweg Storm surge barrier

North Sea

Rotterdam


First planned in 1953.Completed in 1999.Some statistical data:

Each barrier wall has the height of one Eifel Tour, and weighs twice as much.Decision are taken 24 hrs before actual closure,Reversible until 3 hrs before closure.

Fully mechanised -software controlled - decision procedure.


(where ‘fully’ means FULLYFULLY’).



North Sea

Rotterdam

‘BESW’

‘BOS’

North Wall

South Wall


The Storm surge barrier

System consists of distributed

components: north wall,south wall,various hydraulic parts, engines, etc. BOS (‘beslissing & ondersteunend systeem’)

knows the environmental conditions;takes decisions, based on the available data;

BESW (‘besturingssysteem waterweg’)knows & controls the barrier;carries out commands of BOS;reports status information to BOS;


The design philosophy

take

n f

rom

‘M

inis

teri

e v

an V

erk

eer

en W

ate

rsta

at’

‘BBI’(BOS-BESW Interface)



Budget issues

Total costs> 500 million €;

Costs for software< 10 million € (< 2%)

Control software (‘BBI’) developed mainly by CMG.

Formal specification techniques used:

Z

Promela (academic SDL variant, nicer)

Experience (in a nutshell):

Difficult to learn, but pays off enormously.



BBI main components

BOS is informed every 10 minute about water, wind and weather status and forecast

computes anticipated water level;

assesses the anticipation relative to the closing criteria;

if needed

takes a decision (to close or … to open!);

instructs the BESW to implement the decision.



BBI main components

BESW controlswater levels in docks;

opening/closing of dock gates;

moving of barrier walls;

sinking and refloating of barrier walls;…

BESW implements the BOS instructions. BOS and BESW are about 300

mtrs apart, and interact via redundant bidirectional channel pairs.

In particular, they exchange ‘heartbeat’ signals every 30 sec to indicate ‘I am alive’.


Some fragments of the BBI in SDL

block BOS BOS

[status,stop,close,…]

[data]ENV

[curr]

BOS2BESW BESW2BOSsystem BBI

BOS

[status, stop, close,…]

[data, emergency]

[curr]

ENV

BOS2BESW

BESW2BOS

BESW

SIGNAL status, stop, close, curr, …;

[curr]

block BESW

BESW

NORTH

[closed,…]

SOUTH

[close,…][close,…]

[closed,…]

[status,stop,close,…]BOS2BESW BESW2BOS

SIGNAL close, closed, …;


BESW process fragment in SDL

process BOS

S_active:=ffS_ready :=tt

Closing

closed FROM SOUTH

curr(active,ready,stopped)

-

status closed

FROM NORTH

S_active

-

N_active:=ffN_ready :=tt

N_active

-

active := S_active && N_activeready := S_ready && N_readystopped:= S_stopped && N_stopped

-

stop

S_active

S_active := ffS_stopped:= tt

N_active

N_active := ffN_stopped:= tt

…

…

tt

tt

ff

ff

DCL N_active, S_active, active Boolean; N_ready, S_ready, ready Boolean; N_stopped, S_stopped, stopped Boolean; …;


-

*

BOS process fragments in SDL

process BOS

Checking

status

Waiting

NONE


ready || stopped

tt

Stable

Idle

close

Checking

data(…)

…

ff

emergency

stop

DCL active Boolean; ready Boolean; stopped Boolean; …;


Problems?

Well, here is the intended behaviour.

That’s how it should be.

Good!

Yahoo!


58Holger Hermanns Formal Methods for Sof tware Engineeiring - Lecture 10

BESW process fragment in SDL

process BOS

S_active:=ffS_ready :=tt

Closing

closed FROM SOUTH


-

statusclosed

FROM NORTH

S_active

-

N_active:=ffN_ready :=tt

N_active

-

active := S_active && N_activeready := S_ready && N_readystopped:= S_stopped && N_stopped

-

stop

S_active

S_active := ffS_stopped:= tt

N_active

N_active := ffN_stopped:= tt

…

…

tt

tt

ff

ff

DCLN_active, S_active, active Boolean;N_ready, S_ready, ready Boolean;N_stopped, S_stopped, stopped Boolean;…;

Problem!

The system may get stuck if a `stop’ message arrives at the BESW while the gates are closing.


Here is the (almost) original MSC,

reported by Pim Kars in November 1998.

It was found with the model checker SPIN.


Here is another MSC,

reporting a timing violation problem

discovered by Theo Ruys a little later.

This design error has to do with

the heartbeat signals and

maximally anticipated delays when links fail.

I cannot explain this problem,

without adding too much detail.

What I can tell you:

The designers implemented a solution

different from the one proposed.

The implemented solution was never verified.


Storm surge barrier: Results

Z was used for specifying the functions performed by processes; syntax- and type-checking was done with the ZTC tool;was found

very usefulto allow a too great deal of freedom and to offer little structure for the style in which it is to be used;

was equipped with a common ‘style’ (comparable to a coding-standard) for using Z within the project, containing heuristics and pragmatic rules for its use.


Storm surge barrier: Results

Promela:used for modelling the interaction between processes and to the “outside” world. limited to the verification of standard properties such as the absence of deadlock and successful because 1. it helped in reducing defects and 2. it helped in detecting defects early in the development process (reduces the effort and cost required in later stages of development.)

This work was done by Pim Kars, Wil Janssen, Theo Ruys, Jan Tretmans, Job Zwiers and Ed Brinksma.


The BBI experience in the literature

[Tretmans,Wijbrans,Chaudron 2001]



A real application.





Wrap up.


System and software design cycle

Requirements

Engineering

Conceptual

Design

Detailled

Design

Maintenance

Coding

Testing

Design

Validation


Computer-assistance in design and validation

Requirements

Engineering

Conceptual

Design

Detailled

Design

Maintenance

Coding

Testing

Design

Validation

Early phases:(Graphical) Specification formalisms:

SDL, FSP, Z, Promela, UML

Later Phases: Implementation

Code Generation

Early phases:Verification

Later Phases: Test Generation & Execution, Debugging


Computer-assisted system validationDoes a design D satisfy a requirement ?

Techniques:Theorem proving

Strategy: generate a formal proof that D satisfies .

Applicable if design D can be represented in some adequate mathematical theory.

Model checkingStrategy: check systematically and exhaustively whether each reachable state in D satisfies .

Applicable if the behaviour of D can be finitely represented.

Simulation, or TestingStrategy: Check whether holds on some executions of D.

Applicable if D is executable.


Testing

Test whether it works ……


Testing

to check the quality of an objectby performing experimentsin a controlled way

Software Testing: testing the quality of a software product


Testing

Testing is:importantmuch practiced30% - 50% of project effort - at leastexpensivetime criticalnot constructive(but sadistic?)

But also:ad-hoc, manual, error-pronehardly theory / researchno attention in curriculanot cool :“if you’re a bad programmer you might be a tester”

Attitude is changing:more awarenessmore professionalism


construction:implementation

processchecking:testingprocess

implementation

specification Testing functional behaviour

of black-box implementation

with respect to a specification

Specification Based Functional Testing


construction:implementation

processchecking:testingprocess

implementation

formalspecification

Testing functional behaviour

of black-box implementation

with respect to a specificationin a formal language

based on a formal definition

of conformance

Specification Based Functional Testingwith formal methods


Testing based on formal specification

Testing with respect to a formal specification

Precise, formal definition of correctness:

good and unambiguous basis for testing

Formal validation of tests

Algorithmic derivation of tests:

tools for automatic test generation

Allows to define measures expressing coverage

and quality of testing


Labelled Transition System based Testing

Testing theory based on LTS

Specifications, implementations, and testsall modelled as LTS

ConReq

ConConf

Discon Data

Discon


Labelled Transition System based Testing

? x (x >= 0)! x

? x (x < 0)

? y

! -x? x (x >= 0)! x

? x (x < 0)

? y

You have seen such transition systems already in the SDL context, remember? (but their transitions were denoted slightly different there).

This is just another way of expressingnon-persistent input a lá SDL.


Algorithm

To generate a test case from an IOTS with initial state s.

1 end test case

PASS

Apply the following steps recursively, non-deterministically

2 supply input

supply ?a

t(S after ?a)

Test Generation Algorithm

3 observe output

FAIL

t(S after !x)

FAIL

allowed outputs !x

forbidden outputs

!y


specification

test

! 9

! 4

? -2? 2

PASS PASS

otherwise

FAIL

PASS

otherwise

? 3? -3

FAIL

? x (x >= 0)

! x

? x (x < 0)

! -x

To cope with non-deterministic

behaviour, tests are not linear traces,

but trees

Test Generation Example

Equation solver for y2=x

Let specification play against

implementation, with input

and output reversed


? x (x >= 0)

! x

? x (x < 0)

! -x

? y

implementation

test

! 9

! 4

? -2? 2

PASS PASS

otherwise

FAIL

PASS

otherwise

? 3? -3

FAIL

Test Execution Example


A Test Tool: TorXOn-the-fly test generation and test execution

Correctness criterion: ‘ioco’

developed within the ‘Cote-de-Resyste’ project

Specification languages:

LOTOS, FSP

Promela (SDL inspired, nicer)

TorX

nextinput

specification

IUTobserveoutput

offerinput

checkoutput

passfailinconclusive

user:manualautomatic

specification

great stuff

pretty successful

(FMT, Philips,

Lucent,...)

Implementation

under

Test


explorer

primer

driver adapter

IUTIUTIUT bits

bytes

states

transitions

abstract

actions

transition

? x (x >= 0)

! x

? x (x < 0)

! -x

specification implementation

? x (x >= 0)

! x

? x (x < 0)

? x

On-The-Fly Testing

Choose

! x (x < 0)

! x (x >= 0)Choice

! 9

Abstract action

! 9

Concrete action

! 00001001

specification


explorer

primer

driver adapter

IUTIUTIUT bits

bytes

states

transitions

abstract

actions

transition

? x (x >= 0)

! x

? x (x < 0)

! -x

specification implementation

? x (x >= 0)

! x

? x (x < 0)

? x

On-The-Fly Testing

Concrete action

? 00000011

Abstract action

? 3

Action

? 9

Action

? x


TorX



A real application.





Wrap up.


Interpay ‘’Rekeningrijden’’ Highway Tolling System

Appareantly simple protocol

Parallellism:many cars at the same time

Real-time constraints

Encryption


PaymentBox

RoadSideEquipment

OnboardUnit

UDP/IPWireless

Highway Tolling System


specification

Payment Box TorX

PaymentBox

System under Test

‘’Rekeningrijden’’: Test Architecture

+UDP/IP

UDP/IP


‘’Rekeningrijden’’: Results

Results:errors detected during model checking (design error)errors detected during testing (coding error)

Automated testing:

beneficial: high volume and reliabilityvery flexible: adaptation and many configurations

Real-time:How to cope with real time constraints ?Efficient computation for on-the-fly testing ?

Real life:How to cope with fast-moving managers? (Roel-Pieper-Effect)

This work was done by Axel Belinfante, René De Vries, Jan Feenstra and Jan Tretmans.



A real application.



Model construction and model checking beyond what you have seen in the entire set of lectures


Wrap up.


Model Construction

What’s the obstacle?


I deleted some slides here, which I did not show due to

time constraints, and because they are a bit off track.

The slides were motivating ‘compositionality’ as the

main principle to control state space sizes and

verifcation issues. I keep here one technical slide on that issue

(see next slide).


Compositionality, in more formal terms

Given an equivalence relation on processes P, Q, R, ... (LTSs

(S,L,,s) )

and some operators to compose/manipulate processes, op (P,R), op’(P), ...

you want:

if P and Q are equivalent,

then

op(P,R) and op(Q,R) are equivalent,

op’(P,R) and op’(Q) are equivalent,

...

whatever R may be.

R) || Q( R) || (P : R Q P

an example

(observation equivalence, also trace equivalence

but not trace-and-deadlock equivalence)


Model construction features: What’ we haven’t told you

process Controller (int y) {

choose{

:: when (y > 0) {

fail;

Controller (y-1);

}

:: when (y<2) {

throw sleep;

}

:: when (y=0){...}

}

}

loop{

try{

par {

::Gyro()

::Gyro()

::Gyro()

::Gyro()

::Gyro()

::Gyro()

::Controller(6);

}

} catch sleep {

prepare;launch; repair;

}

}

For instance:

Exception Handling

Real time (important)

Randomness (cool)

Priorities (tough)

...

Main criterion for a reasonable set up:

compositionality



A real application.





Wrap up.


Demo of UPPAAL goes here.



A real application.





Wrap up.


Verification vs. Testing

23Holger Hermanns Formal Methods for Sof tware Engineeiring - Lecture 10

Computer- assisted system validationDoes a design D satisf y a requirement ?

Techniques:Theorem proving

Strategy: generate a f ormal proof that D satisfi es .

Applicable if design D can be represented in some adequate mathematical theory.

Model checkingStrategy: check systematically and exhaustively whether each reachable state in D satisfi es .

Applicable if the behaviour of D can be fi nitely represented.

Simulation, or TestingStrategy: Check whether holds on some executions of D.

Applicable if D is executable.

verification


formal world

concrete world

Verification is only as good as the validity of the model

on which it is based

Verification is only as good as the validity of the model

on which it is based

Verification vs. Testing

Verification:(model checking, theorem proving)

formal manipulationprove propertiesperformed on model

Testing :experimentationshow errorconcrete system

Testing can only show the presence of errors, not their

absence

Testing can only show the presence of errors, not their

absence


What was this whole lecture all about

Make you alert about concurrency and its difficulties

Tell you about the principal model(s) of concurrency

Give you a flavor of important specification languages

Tell you about the principles behind such languages

Let you play with Z, FSP, and SDL


What it was not about

Deriving implementations from a formal specification ‘protocol engineering’ (Luis Pires)

Model checking a formal specification ‘system validation’ (Joost-Pieter Katoen)

Formal specification-based testing ‘testing techniques’ (Ed Brinksma)

Web page design

But it prepared you for these lectures.


A few days into the mission, not long after Pathfinder started gathering meteorological data, the spacecraft began experiencing total system resets, each resulting in losses of data. Curious?

Yet another example: Mars Pathfinder

Holger Hermanns Formal Methods for Software Engineering Part III: Applications of Formal Methods...

Documents

Transcript of Holger Hermanns Formal Methods for Software Engineering Part III: Applications of Formal Methods...