Formal Methods in Software Engineering Applications of Formal Methods Lecture 31.

of 59 /59
Formal Methods in Software Engineering Applications of Formal Methods Lecture 31

Embed Size (px)

Transcript of Formal Methods in Software Engineering Applications of Formal Methods Lecture 31.

  • Formal Methods in Software Engineering

    Applications of Formal MethodsLecture 31

    Formal Methods in Software Engineering - Lecture 31

    *This PartWe apply the concepts, methods and tools you learnt to love in contexts that are relatively close to what the people out there are facing.

    In this lecture I show you what they are facing,and I round off the entire lecture series. Assumptions for today:Nothing particular.

    Formal Methods in Software Engineering - Lecture 31

    *Contents of this lectureA real application.

    Testing based on formal methods.

    Another real application.

    Model construction and model checking beyond what you have seen in this entire set of lectures.

    A third, very real application.

    Wrap up.

    Formal Methods in Software Engineering - Lecture 31

    *Whats this?

    Formal Methods in Software Engineering - Lecture 31

    *Nieuwe Waterweg Storm surge barrierNorth Sea Rotterdam

    Formal Methods in Software Engineering - Lecture 31

    *First planned in 1953.Completed in 1999.Some statistical data:Each barrier wall has the height of one Eifel Tour, and weighs twice as much.Decision are taken 24 hrs before actual closure,Reversible until 3 hrs before closure.

    Fully mechanised -software controlled - decision procedure.Nieuwe Waterweg Storm surge barrier(where fully means FULLY).

    Formal Methods in Software Engineering - Lecture 31

    *Nieuwe Waterweg Storm surge barrierNorth Sea Rotterdam BESWBOSNorth Wall South Wall

    Formal Methods in Software Engineering - Lecture 31

    *The Storm surge barrierSystem consists of distributed components: north wall,south wall,various hydraulic parts, engines, etc.

    BOS (beslissing & ondersteunend systeem) knows the environmental conditions;takes decisions, based on the available data;BESW (besturingssysteem waterweg)knows & controls the barrier;carries out commands of BOS;reports status information to BOS;

    Formal Methods in Software Engineering - Lecture 31

    *The design philosophytaken from Ministerie van Verkeer en WaterstaatBBI(BOS-BESW Interface)

    Formal Methods in Software Engineering - Lecture 31

    *The Storm surge barrierBudget issuesTotal costs> 500 million ;Costs for software< 10 million (< 2%)

    Control software (BBI) developed mainly by CMG.Formal specification techniques used: Z Promela (academic SDL variant, nicer)Experience (in a nutshell): Difficult to learn, but pays off enormously.

    Formal Methods in Software Engineering - Lecture 31

    *The Storm surge barrierBBI main componentsBOS is informed every 10 minute about water, wind and weather status and forecast

    computes anticipated water level;assesses the anticipation relative to the closing criteria;if needed takes a decision (to close or to open!);instructs the BESW to implement the decision.

    Formal Methods in Software Engineering - Lecture 31

    *The Storm surge barrierBBI main componentsBESW controlswater levels in docks;opening/closing of dock gates;moving of barrier walls;sinking and refloating of barrier walls;BESW implements the BOS instructions.

    BOS and BESW are about 300 mtrs apart, and interact via redundant bidirectional channel pairs. In particular, they exchange heartbeat signals every 30 sec to indicate I am alive.

    Formal Methods in Software Engineering - Lecture 31

    *Some fragments of the BBI in SDLsystem BBI BOS[status, stop, close,][data, emergency][curr]ENV

    BOS2BESW

    BESW2BOS BESWSIGNAL status, stop, close, curr, ;

    Formal Methods in Software Engineering - Lecture 31

    *BESW process fragment in SDLprocess BOSS_active:=ffS_ready :=ttClosingcurr(active,ready,stopped)--N_active:=ffN_ready :=tt-active := S_active && N_activeready := S_ready && N_readystopped:= S_stopped && N_stopped-S_activeS_active := ffS_stopped:= ttN_activeN_active := ffN_stopped:= ttttttffffDCL N_active, S_active, active Boolean; N_ready, S_ready, ready Boolean; N_stopped, S_stopped, stopped Boolean; ;

    Formal Methods in Software Engineering - Lecture 31

    *BOS process fragments in SDLprocess BOSCheckingstatusWaitingready || stoppedttStableIdlecloseCheckingffstopDCL active Boolean; ready Boolean; stopped Boolean; ;

    Formal Methods in Software Engineering - Lecture 31

    *Problems?Well, here is the intended behaviour.

    Thats how it should be.

    Good!

    Yahoo!

    Formal Methods in Software Engineering - Lecture 31

    *Problem!The system may get stuck if a `stop message arrives at the BESW while the gates are closing.

    Holger Hermanns Formal Methods for Software Engineeiring - Lecture 10

    BESW process fragment in SDL

    process BOS

    S_active:=ffS_ready :=tt

    Closing

    curr(active,ready,stopped)

    -

    -

    N_active:=ffN_ready :=tt

    -

    active := S_active && N_activeready := S_ready && N_readystopped:= S_stopped && N_stopped

    -

    S_active

    S_active := ffS_stopped:= tt

    N_active

    N_active := ffN_stopped:= tt

    tt

    tt

    ff

    ff

    DCL N_active, S_active, active Boolean; N_ready, S_ready, ready Boolean; N_stopped, S_stopped, stopped Boolean; ;

    Dr.-Ing Holger Hermanns, Universiteit Twente

    Formal Methods in Software Engineering - Lecture 31

    *Here is the (almost) original MSC, reported by Pim Kars in November 1998. It was found with the model checker SPIN.

    Formal Methods in Software Engineering - Lecture 31

    *Here is another MSC, reporting a timing violation problem discovered by Theo Ruys a little later. This design error has to do with the heartbeat signals and maximally anticipated delays when links fail. I cannot explain this problem, without adding too much detail. What I can tell you:The designers implemented a solution different from the one proposed. The implemented solution was never verified.

    Formal Methods in Software Engineering - Lecture 31

    *Storm surge barrier: ResultsZ was used for specifying the functions performed by processes; syntax- and type-checking was done with the ZTC tool;was found very usefulto allow a too great deal of freedom and to offer little structure for the style in which it is to be used; was equipped with a common style (comparable to a coding-standard) for using Z within the project, containing heuristics and pragmatic rules for its use.

    Formal Methods in Software Engineering - Lecture 31

    *Storm surge barrier: ResultsPromela:used for modelling the interaction between processes and to the outside world. limited to the verification of standard properties such as the absence of deadlock and successful because it helped in reducing defects and it helped in detecting defects early in the development process (reduces the effort and cost required in later stages of development.)

    This work was done by Pim Kars, Wil Janssen, Theo Ruys, Jan Tretmans, Job Zwiers and Ed Brinksma.

    Formal Methods in Software Engineering - Lecture 31

    *

    The BBI experience in the literature[Tretmans,Wijbrans,Chaudron 2001]

    Formal Methods in Software Engineering - Lecture 31

    *Contents of this lectureA real application.

    Testing based on formal methods.

    Another real application.

    Model construction and model checking beyond what you have seen in this entire set of lectures.

    A third, very real application.

    Wrap up.

    Formal Methods in Software Engineering - Lecture 31

    *System and software design cycleDesignValidation

    Formal Methods in Software Engineering - Lecture 31

    *Computer-assistance in design and validationDesignValidationEarly phases:(Graphical) Specification formalisms:SDL, FSP, Z, Promela, UML

    Later Phases: Implementation Code Generation Early phases:Verification

    Later Phases: Test Generation & Execution, Debugging

    Formal Methods in Software Engineering - Lecture 31

    *Computer-assisted system validationDoes a design D satisfy a requirement ?Techniques:Theorem provingStrategy: generate a formal proof that D satisfies . Applicable if design D can be represented in some adequate mathematical theory.

    Model checkingStrategy: check systematically and exhaustively whether each reachable state in D satisfies .Applicable if the behaviour of D can be finitely represented.

    Simulation, or TestingStrategy: Check whether holds on some executions of D.Applicable if D is executable.

    Formal Methods in Software Engineering - Lecture 31

    *TestingTest whether it works

    Formal Methods in Software Engineering - Lecture 31

    *Testingto check the quality of an objectby performing experimentsin a controlled waySoftware Testing: testing the quality of a software product

    Formal Methods in Software Engineering - Lecture 31

    *TestingTesting is:importantmuch practiced30% - 50% of project effort - at leastexpensivetime criticalnot constructive (but sadistic?)But also:ad-hoc, manual, error-pronehardly theory / researchno attention in curriculanot cool : if youre a bad programmer you might be a testerAttitude is changing:more awarenessmore professionalism

    Formal Methods in Software Engineering - Lecture 31

    *construction:implementation processchecking:testingprocessTesting functional behaviourof black-box implementation with respect to a specificationSpecification Based Functional Testing

    Formal Methods in Software Engineering - Lecture 31

    *construction:implementation processchecking:testingprocessformal specificationTesting functional behaviourof black-box implementationwith respect to a specification in a formal language based on a formal definitionof conformance

    Specification Based Functional Testingwith formal methods

    Formal Methods in Software Engineering - Lecture 31

    *Testing based on formal specificationTesting with respect to a formal specificationPrecise, formal definition of correctness: good and unambiguous basis for testingFormal validation of testsAlgorithmic derivation of tests: tools for automatic test generationAllows to define measures expressing coverage and quality of testing

    Formal Methods in Software Engineering - Lecture 31

    *Labelled Transition System based TestingTesting theory based on LTS

    Specifications, implementations, and tests all modelled as LTS

    Formal Methods in Software Engineering - Lecture 31

    *Labelled Transition System based Testing

    Uses a common variation of labelled transition system: Input-Output Transition System - IOTSdistinction between outputs ! and always-enabled inputs ?

    IOTS with variables - equation solver for y2 =xYou have seen such transition systems already in the SDL context, remember? (but their transitions were denoted slightly different there).This is just another way of expressingnon-persistent input a l SDL.

    Formal Methods in Software Engineering - Lecture 31

    *AlgorithmTo generate a test case from an IOTS with initial state s.Apply the following steps recursively, non-deterministicallyTest Generation Algorithm

    Formal Methods in Software Engineering - Lecture 31

    *specificationtest? x (x >= 0)! x? x (x < 0)! -xTo cope with non-deterministic behaviour, tests are not linear traces, but treesTest Generation ExampleEquation solver for y2=xLet specification play against implementation, with input and output reversed

    Formal Methods in Software Engineering - Lecture 31

    *implementationtestTest Execution Example

    Formal Methods in Software Engineering - Lecture 31

    *A Test Tool: TorXOn-the-fly test generation and test executionCorrectness criterion: ioco developed within the Cote-de-Resyste projectSpecification languages: LOTOS, FSP Promela (SDL inspired, nicer)great stuffpretty successful (FMT, Philips, Lucent,...)ImplementationunderTest

    Formal Methods in Software Engineering - Lecture 31

    *On-The-Fly TestingChoose! x (x < 0)! x (x >= 0)Choice! 9Abstract action! 9Concrete action! 00001001

    Formal Methods in Software Engineering - Lecture 31

    *implementationOn-The-Fly TestingConcrete action? 00000011Abstract action? 3Action? 9Action? x

    Formal Methods in Software Engineering - Lecture 31

    *TorX

    Formal Methods in Software Engineering - Lecture 31

    *Contents of this lectureA real application.

    Testing based on formal methods.

    Another real application.

    Model construction and model checking beyond what you have seen in this entire set of lectures.

    A third, very real application.

    Wrap up.

    Formal Methods in Software Engineering - Lecture 31

    *Interpay Rekeningrijden Highway Tolling SystemAppareantly simple protocolParallellism:many cars at the same timeReal-time constraintsEncryption

    Formal Methods in Software Engineering - Lecture 31

    *PaymentBox RoadSideEquipmentOnboardUnitUDP/IPWirelessHighway Tolling System

    Formal Methods in Software Engineering - Lecture 31

    *specificationPayment Box

    TorXPayment BoxSystem under TestRekeningrijden: Test Architecture+UDP/IP

    Formal Methods in Software Engineering - Lecture 31

    *Rekeningrijden: ResultsResults:errors detected during model checking (design error)errors detected during testing (coding error)Automated testing:beneficial: high volume and reliabilityvery flexible: adaptation and many configurationsReal-time:How to cope with real time constraints ?Efficient computation for on-the-fly testing ?Real life:How to cope with fast-moving managers? (Roel-Pieper-Effect)

    This work was done by Axel Belinfante, Ren De Vries, Jan Feenstra and Jan Tretmans.

    Formal Methods in Software Engineering - Lecture 31

    *Contents of this lectureA real application.

    Testing based on formal methods.

    Another real application.

    Model construction and model checking beyond what you have seen in the entire set of lectures

    A third, very real application.

    Wrap up.

    Formal Methods in Software Engineering - Lecture 31

    *Model ConstructionWhats the obstacle?

    Formal Methods in Software Engineering - Lecture 31

    *I deleted some slides here, which I did not show due to time constraints, and because they are a bit off track.

    The slides were motivating compositionality as the main principle to control state space sizes and verifcation issues.

    I keep here one technical slide on that issue (see next slide).

    Formal Methods in Software Engineering - Lecture 31

    *

    Compositionality, in more formal termsGiven an equivalence relation on processes P, Q, R, ... (LTSs (S,L,,s) ) and some operators to compose/manipulate processes, op (P,R), op(P), ...you want: if P and Q are equivalent, then op(P,R) and op(Q,R) are equivalent, op(P,R) and op(Q) are equivalent,... whatever R may be. (observation equivalence, also trace equivalence but not trace-and-deadlock equivalence)

    Formal Methods in Software Engineering - Lecture 31

    *Model construction features: What we havent told youprocess Controller (int y) { choose{:: when (y > 0) {fail; Controller (y-1);}:: when (y