Formal Methods in Software Engineering Applications of Formal Methods Lecture 31.
-
Upload
primrose-lang -
Category
Documents
-
view
219 -
download
4
Transcript of Formal Methods in Software Engineering Applications of Formal Methods Lecture 31.
Formal Methods in Software Engineering
Applications of Formal Methods
Lecture 31
Formal Methods in Software Engineering - Lecture 31 2
This Part
We apply the concepts, methods and tools you learnt to love in contexts that are relatively close to what the people out there are facing.
In this lecture I show you what they are facing,
and I round off the entire lecture series.
Assumptions for today:
Nothing particular.
Formal Methods in Software Engineering - Lecture 31 3
Contents of this lecture
A real application.
Testing based on formal methods.
Another real application.
Model construction and model checking beyond what you have seen in this entire set of lectures.
A third, very real application.
Wrap up.
Formal Methods in Software Engineering - Lecture 31 4
What’s this?
Formal Methods in Software Engineering - Lecture 31 5
Nieuwe Waterweg Storm surge barrier
North Sea
Rotterdam
Formal Methods in Software Engineering - Lecture 31 6
First planned in 1953.Completed in 1999.Some statistical data:
Each barrier wall has the height of one Eifel Tour, and weighs twice as much.Decision are taken 24 hrs before actual closure,Reversible until 3 hrs before closure.
Fully mechanised -software controlled - decision procedure.
Nieuwe Waterweg Storm surge barrier
(where ‘fully’ means FULLYFULLY’).
Formal Methods in Software Engineering - Lecture 31 7
Nieuwe Waterweg Storm surge barrier
North Sea
Rotterdam
‘BESW’
‘BOS’
North Wall
South Wall
Formal Methods in Software Engineering - Lecture 31 8
The Storm surge barrier
System consists of distributed
components: north wall,south wall,various hydraulic parts, engines, etc. BOS (‘beslissing & ondersteunend systeem’)
knows the environmental conditions;takes decisions, based on the available data;
BESW (‘besturingssysteem waterweg’)knows & controls the barrier;carries out commands of BOS;reports status information to BOS;
Formal Methods in Software Engineering - Lecture 31 9
The design philosophy
take
n f
rom
‘M
inis
teri
e v
an V
erk
eer
en W
ate
rsta
at’
‘BBI’(BOS-BESW Interface)
Formal Methods in Software Engineering - Lecture 31 10
The Storm surge barrier
Budget issues
Total costs> 500 million €;
Costs for software< 10 million € (< 2%)
Control software (‘BBI’) developed mainly by CMG.
Formal specification techniques used:
Z
Promela (academic SDL variant, nicer)
Experience (in a nutshell):
Difficult to learn, but pays off enormously.
Formal Methods in Software Engineering - Lecture 31 11
The Storm surge barrier
BBI main components
BOS is informed every 10 minute about water, wind and weather status and forecast
computes anticipated water level;
assesses the anticipation relative to the closing criteria;
if needed
takes a decision (to close or … to open!);
instructs the BESW to implement the decision.
Formal Methods in Software Engineering - Lecture 31 12
The Storm surge barrier
BBI main components
BESW controlswater levels in docks;
opening/closing of dock gates;
moving of barrier walls;
sinking and refloating of barrier walls;…
BESW implements the BOS instructions. BOS and BESW are about 300
mtrs apart, and interact via redundant bidirectional channel pairs.
In particular, they exchange ‘heartbeat’ signals every 30 sec to indicate ‘I am alive’.
Formal Methods in Software Engineering - Lecture 31 13
Some fragments of the BBI in SDL
block BOS BOS
[status,stop,close,…]
[data]ENV
[curr]
BOS2BESW BESW2BOSsystem BBI
BOS
[status, stop, close,…]
[data, emergency]
[curr]
ENV
BOS2BESW
BESW2BOS
BESW
SIGNAL status, stop, close, curr, …;
[curr]
block BESW
BESW
NORTH
[closed,…]
SOUTH
[close,…][close,…]
[closed,…]
[status,stop,close,…]BOS2BESW BESW2BOS
SIGNAL close, closed, …;
Formal Methods in Software Engineering - Lecture 31 14
BESW process fragment in SDL
process BOS
S_active:=ffS_ready :=tt
Closing
closed FROM SOUTH
curr(active,ready,stopped)
-
status closed
FROM NORTH
S_active
-
N_active:=ffN_ready :=tt
N_active
-
active := S_active && N_activeready := S_ready && N_readystopped:= S_stopped && N_stopped
-
stop
S_active
S_active := ffS_stopped:= tt
N_active
N_active := ffN_stopped:= tt
…
…
tt
tt
ff
ff
DCL N_active, S_active, active Boolean; N_ready, S_ready, ready Boolean; N_stopped, S_stopped, stopped Boolean; …;
Formal Methods in Software Engineering - Lecture 31 15
-
*
BOS process fragments in SDL
process BOS
Checking
status
Waiting
NONE
curr(active,ready,stopped)
ready || stopped
tt
Stable
Idle
close
Checking
data(…)
…
ff
emergency
stop
DCL active Boolean; ready Boolean; stopped Boolean; …;
Formal Methods in Software Engineering - Lecture 31 16
Problems?
Well, here is the intended behaviour.
That’s how it should be.
Good!
Yahoo!
Formal Methods in Software Engineering - Lecture 31 17
58Holger Hermanns Formal Methods for Sof tware Engineeiring - Lecture 10
BESW process fragment in SDL
process BOS
S_active:=ffS_ready :=tt
Closing
closed FROM SOUTH
curr(active,ready,stopped)
-
statusclosed
FROM NORTH
S_active
-
N_active:=ffN_ready :=tt
N_active
-
active := S_active && N_activeready := S_ready && N_readystopped:= S_stopped && N_stopped
-
stop
S_active
S_active := ffS_stopped:= tt
N_active
N_active := ffN_stopped:= tt
…
…
tt
tt
ff
ff
DCLN_active, S_active, active Boolean;N_ready, S_ready, ready Boolean;N_stopped, S_stopped, stopped Boolean;…;
Problem!
The system may get stuck if a `stop’ message arrives at the BESW while the gates are closing.
Formal Methods in Software Engineering - Lecture 31 18
Here is the (almost) original MSC,
reported by Pim Kars in November 1998.
It was found with the model checker SPIN.
Formal Methods in Software Engineering - Lecture 31 19
Here is another MSC,
reporting a timing violation problem
discovered by Theo Ruys a little later.
This design error has to do with
the heartbeat signals and
maximally anticipated delays when links fail.
I cannot explain this problem,
without adding too much detail.
What I can tell you:
The designers implemented a solution
different from the one proposed.
The implemented solution was never verified.
Formal Methods in Software Engineering - Lecture 31 20
Storm surge barrier: Results
Z was used for specifying the functions performed by processes; syntax- and type-checking was done with the ZTC tool;was found
very usefulto allow a too great deal of freedom and to offer little structure for the style in which it is to be used;
was equipped with a common ‘style’ (comparable to a coding-standard) for using Z within the project, containing heuristics and pragmatic rules for its use.
Formal Methods in Software Engineering - Lecture 31 21
Storm surge barrier: Results
Promela:used for modelling the interaction between processes and to the “outside” world. limited to the verification of standard properties such as the absence of deadlock and successful because 1. it helped in reducing defects and 2. it helped in detecting defects early in the development process (reduces the effort and cost required in later stages of development.)
This work was done by Pim Kars, Wil Janssen, Theo Ruys, Jan Tretmans, Job Zwiers and Ed Brinksma.
Formal Methods in Software Engineering - Lecture 31 22
The BBI experience in the literature
[Tretmans,Wijbrans,Chaudron 2001]
Formal Methods in Software Engineering - Lecture 31 23
Contents of this lecture
A real application.
Testing based on formal methods.
Another real application.
Model construction and model checking beyond what you have seen in this entire set of lectures.
A third, very real application.
Wrap up.
Formal Methods in Software Engineering - Lecture 31 24
System and software design cycle
Requirements
Engineering
Conceptual
Design
Detailled
Design
Maintenance
Coding
Testing
Design
Validation
Formal Methods in Software Engineering - Lecture 31 25
Computer-assistance in design and validation
Requirements
Engineering
Conceptual
Design
Detailled
Design
Maintenance
Coding
Testing
Design
Validation
Early phases:(Graphical) Specification formalisms:
SDL, FSP, Z, Promela, UML
Later Phases: Implementation
Code Generation
Early phases:Verification
Later Phases: Test Generation & Execution, Debugging
Formal Methods in Software Engineering - Lecture 31 26
Computer-assisted system validationDoes a design D satisfy a requirement ?
Techniques:Theorem proving
Strategy: generate a formal proof that D satisfies .
Applicable if design D can be represented in some adequate mathematical theory.
Model checkingStrategy: check systematically and exhaustively whether each reachable state in D satisfies .
Applicable if the behaviour of D can be finitely represented.
Simulation, or TestingStrategy: Check whether holds on some executions of D.
Applicable if D is executable.
Formal Methods in Software Engineering - Lecture 31 27
Testing
Test whether it works ……
Formal Methods in Software Engineering - Lecture 31 28
Testing
to check the quality of an objectby performing experimentsin a controlled way
Software Testing: testing the quality of a software product
Formal Methods in Software Engineering - Lecture 31 29
Testing
Testing is:importantmuch practiced30% - 50% of project effort - at leastexpensivetime criticalnot constructive(but sadistic?)
But also:ad-hoc, manual, error-pronehardly theory / researchno attention in curriculanot cool :“if you’re a bad programmer you might be a tester”
Attitude is changing:more awarenessmore professionalism
Formal Methods in Software Engineering - Lecture 31 30
construction:implementation
processchecking:testingprocess
implementation
specification Testing functional behaviour
of black-box implementation
with respect to a specification
Specification Based Functional Testing
Formal Methods in Software Engineering - Lecture 31 31
construction:implementation
processchecking:testingprocess
implementation
formalspecification
Testing functional behaviour
of black-box implementation
with respect to a specificationin a formal language
based on a formal definition
of conformance
Specification Based Functional Testingwith formal methods
Formal Methods in Software Engineering - Lecture 31 32
Testing based on formal specification
Testing with respect to a formal specification
Precise, formal definition of correctness:
good and unambiguous basis for testing
Formal validation of tests
Algorithmic derivation of tests:
tools for automatic test generation
Allows to define measures expressing coverage
and quality of testing
Formal Methods in Software Engineering - Lecture 31 33
Labelled Transition System based Testing
Testing theory based on LTS
Specifications, implementations, and testsall modelled as LTS
ConReq
ConConf
Discon Data
Discon
Formal Methods in Software Engineering - Lecture 31 34
Labelled Transition System based Testing
? x (x >= 0)! x
? x (x < 0)
? y
! -x? x (x >= 0)! x
? x (x < 0)
? y
You have seen such transition systems already in the SDL context, remember? (but their transitions were denoted slightly different there).
This is just another way of expressingnon-persistent input a lá SDL.
Formal Methods in Software Engineering - Lecture 31 35
Algorithm
To generate a test case from an IOTS with initial state s.
1 end test case
PASS
Apply the following steps recursively, non-deterministically
2 supply input
supply ?a
t(S after ?a)
Test Generation Algorithm
3 observe output
FAIL
t(S after !x)
FAIL
allowed outputs !x
forbidden outputs
!y
Formal Methods in Software Engineering - Lecture 31 36
specification
test
! 9
! 4
? -2? 2
PASS PASS
otherwise
FAIL
PASS
otherwise
? 3? -3
FAIL
? x (x >= 0)
! x
? x (x < 0)
! -x
To cope with non-deterministic
behaviour, tests are not linear traces,
but trees
Test Generation Example
Equation solver for y2=x
Let specification play against
implementation, with input
and output reversed
Formal Methods in Software Engineering - Lecture 31 37
? x (x >= 0)
! x
? x (x < 0)
! -x
? y
implementation
test
! 9
! 4
? -2? 2
PASS PASS
otherwise
FAIL
PASS
otherwise
? 3? -3
FAIL
Test Execution Example
Formal Methods in Software Engineering - Lecture 31 38
A Test Tool: TorXOn-the-fly test generation and test execution
Correctness criterion: ‘ioco’
developed within the ‘Cote-de-Resyste’ project
Specification languages:
LOTOS, FSP
Promela (SDL inspired, nicer)
TorX
nextinput
specification
IUTobserveoutput
offerinput
checkoutput
passfailinconclusive
user:manualautomatic
specification
great stuff
pretty successful
(FMT, Philips,
Lucent,...)
Implementation
under
Test
Formal Methods in Software Engineering - Lecture 31 39
explorer
primer
driver adapter
IUTIUTIUT bits
bytes
states
transitions
abstract
actions
transition
? x (x >= 0)
! x
? x (x < 0)
! -x
specification implementation
? x (x >= 0)
! x
? x (x < 0)
? x
On-The-Fly Testing
Choose
! x (x < 0)
! x (x >= 0)Choice
! 9
Abstract action
! 9
Concrete action
! 00001001
specification
Formal Methods in Software Engineering - Lecture 31 40
explorer
primer
driver adapter
IUTIUTIUT bits
bytes
states
transitions
abstract
actions
transition
? x (x >= 0)
! x
? x (x < 0)
! -x
specification implementation
? x (x >= 0)
! x
? x (x < 0)
? x
On-The-Fly Testing
Concrete action
? 00000011
Abstract action
? 3
Action
? 9
Action
? x
Formal Methods in Software Engineering - Lecture 31 41
TorX
Formal Methods in Software Engineering - Lecture 31 42
Contents of this lecture
A real application.
Testing based on formal methods.
Another real application.
Model construction and model checking beyond what you have seen in this entire set of lectures.
A third, very real application.
Wrap up.
Formal Methods in Software Engineering - Lecture 31 43
Interpay ‘’Rekeningrijden’’ Highway Tolling System
Appareantly simple protocol
Parallellism:many cars at the same time
Real-time constraints
Encryption
Formal Methods in Software Engineering - Lecture 31 44
PaymentBox
RoadSideEquipment
OnboardUnit
UDP/IPWireless
Highway Tolling System
Formal Methods in Software Engineering - Lecture 31 45
specification
Payment Box TorX
PaymentBox
System under Test
‘’Rekeningrijden’’: Test Architecture
+UDP/IP
UDP/IP
Formal Methods in Software Engineering - Lecture 31 46
‘’Rekeningrijden’’: Results
Results:errors detected during model checking (design error)errors detected during testing (coding error)
Automated testing:
beneficial: high volume and reliabilityvery flexible: adaptation and many configurations
Real-time:How to cope with real time constraints ?Efficient computation for on-the-fly testing ?
Real life:How to cope with fast-moving managers? (Roel-Pieper-Effect)
This work was done by Axel Belinfante, René De Vries, Jan Feenstra and Jan Tretmans.
Formal Methods in Software Engineering - Lecture 31 47
Contents of this lecture
A real application.
Testing based on formal methods.
Another real application.
Model construction and model checking beyond what you have seen in the entire set of lectures
A third, very real application.
Wrap up.
Formal Methods in Software Engineering - Lecture 31 48
Model Construction
What’s the obstacle?
Formal Methods in Software Engineering - Lecture 31 49
I deleted some slides here, which I did not show due to
time constraints, and because they are a bit off track.
The slides were motivating ‘compositionality’ as the
main principle to control state space sizes and
verifcation issues. I keep here one technical slide on that issue
(see next slide).
Formal Methods in Software Engineering - Lecture 31 50
Compositionality, in more formal terms
Given an equivalence relation on processes P, Q, R, ... (LTSs
(S,L,,s) )
and some operators to compose/manipulate processes, op (P,R), op’(P), ...
you want:
if P and Q are equivalent,
then
op(P,R) and op(Q,R) are equivalent,
op’(P,R) and op’(Q) are equivalent,
...
whatever R may be.
R) || Q( R) || (P : R Q P
an example
(observation equivalence, also trace equivalence
but not trace-and-deadlock equivalence)
Formal Methods in Software Engineering - Lecture 31 51
Model construction features: What’ we haven’t told you
process Controller (int y) {
choose{
:: when (y > 0) {
fail;
Controller (y-1);
}
:: when (y<2) {
throw sleep;
}
:: when (y=0){...}
}
}
loop{
try{
par {
::Gyro()
::Gyro()
::Gyro()
::Gyro()
::Gyro()
::Gyro()
::Controller(6);
}
} catch sleep {
prepare;launch; repair;
}
}
For instance:
Exception Handling
Real time (important)
Randomness (cool)
Priorities (tough)
...
Main criterion for a reasonable set up:
compositionality
Formal Methods in Software Engineering - Lecture 31 52
Contents of this lecture
A real application.
Testing based on formal methods.
Another real application.
Model construction and model checking beyond what you have seen in this entire set of lectures.
A third, very real application.
Wrap up.
Formal Methods in Software Engineering - Lecture 31 53
Demo of UPPAAL goes here.
Formal Methods in Software Engineering - Lecture 31 54
Contents of this lecture
A real application.
Testing based on formal methods.
Another real application.
Model construction and model checking beyond what you have seen in this entire set of lectures.
A third, very real application.
Wrap up.
Formal Methods in Software Engineering - Lecture 31 55
Verification vs. Testing
23Holger Hermanns Formal Methods for Sof tware Engineeiring - Lecture 10
Computer- assisted system validationDoes a design D satisf y a requirement ?
Techniques:Theorem proving
Strategy: generate a f ormal proof that D satisfi es .
Applicable if design D can be represented in some adequate mathematical theory.
Model checkingStrategy: check systematically and exhaustively whether each reachable state in D satisfi es .
Applicable if the behaviour of D can be fi nitely represented.
Simulation, or TestingStrategy: Check whether holds on some executions of D.
Applicable if D is executable.
verification
Formal Methods in Software Engineering - Lecture 31 56
formal world
concrete world
Verification is only as good as the validity of the model
on which it is based
Verification is only as good as the validity of the model
on which it is based
Verification vs. Testing
Verification:(model checking, theorem proving)
formal manipulationprove propertiesperformed on model
Testing :experimentationshow errorconcrete system
Testing can only show the presence of errors, not their
absence
Testing can only show the presence of errors, not their
absence
Formal Methods in Software Engineering - Lecture 31 57
What was this whole lecture all about
Make you alert about concurrency and its difficulties
Tell you about the principal model(s) of concurrency
Give you a flavor of important specification languages
Tell you about the principles behind such languages
Let you play with Z, FSP, and SDL
Formal Methods in Software Engineering - Lecture 31 58
What it was not about
Deriving implementations from a formal specification ‘protocol engineering’ (Luis Pires)
Model checking a formal specification ‘system validation’ (Joost-Pieter Katoen)
Formal specification-based testing ‘testing techniques’ (Ed Brinksma)
Web page design
But it prepared you for these lectures.
Formal Methods in Software Engineering - Lecture 31 59
A few days into the mission, not long after Pathfinder started gathering meteorological data, the spacecraft began experiencing total system resets, each resulting in losses of data. Curious?
Yet another example: Mars Pathfinder