Verifica Automatica via Model Checking
description
Transcript of Verifica Automatica via Model Checking
Verifica Automatica via Model Checking
Enrico TronciDipartimento di Informatica, Università di Roma “La Sapienza”,
Via Salaraia 113, 00198 Roma, Italy, [email protected] http://www.dsi.uniroma1.it/~tronci
May 2006
2
TRAMP
SystemDSS +
Operator + Policy
Communication Channel
Measures
Get measures (with delay,
noise, lost, etc).
Send Command, according to policy
Get commnad (with delay, noise, lost, etc)
Verification Goals. Give evidence of the following
•The interaction between the system (trains, vehiecles, etc), the communication infrastructure (TLC), the control policies in the DSS and the Operator never bring the system to an UNSAFE state;
•The system efficiency is not decreased (is increased) by the supervisory control proposed by the project.
Compute system state and action
3
Model Checking Game
Model Checker(Equivalent to
Exhaustive testing)
Sys(VHDL, Verilog, C, C++
Java, MathLab, Simulink, UML …)
BAD(CTL, CTL*, LTL, PSL, …)
PASSFAILWhat went
wrong …I.e. no sequence of events (states) can possibly lead to an undesired state.
CounterexampleI.e. sequence of events (states) leading to an undesired state.
4
Examples
A few exmples from different domains to illustrate the appraoch
5
A Control System
Controller Gas Turbine(Turbogas)
Disturbances: electric users, param. var, etc
Vrot: Turbine Rotation speedTexh: Exhaust smokes TemperaturePel: Generated Electric PowerPmc: Compressor Pressure
Settings Fuel Valve OpeningFG102
Vrot, Texh, Pel, Pmc
6
Experimental ResultsMAX_D_U(KW/sec)
Reachable States
Rules Fired
Diameter CPU (sec) Result
1000 2,246,328 6,738,984 12904 16988.18 PASS
1750 7,492,389 22,477,167 7423 54012.18 PASS
2500 1,739,719 5,186,047 1533 12548.25 FAIL
5000 36,801 109,015 804 271.77 FAIL
Results on a INTEL Pentium 4, 2GHz Linux PC with 512 MB RAM. Murphi options: -b, -c, --cache, -m350
7
Fail trace: MAX_D_U = 2500 KW/sec10 ms time step (100 Hz sampling frequency)
Electric user demand (KW)
Rotation speed (percentage of max = 22500 rpm)
Allowed range for rotation speed:40-120
8
Fail trace: MAX_D_U = 5000 Kw/sec10 ms time step (100 Hz sampling frequency)
Electric user demand (KW)
Rotation speed (percentage of max = 22500 rpm)
Allowed range for rotation speed:40-120
9
Example
BufferLS HS
Statistically modulated ACK
Low System(e.g. public info)
High System(e.g. private info)
Data Data
ACK
NRL PUMP
The NRL Pump is a special purpose-device that forwards data from a low (security) level system LS to a high (security) level system HS, but not conversely.
Idea: LS ACK delay is probabilistically based on a moving average of HS ACK delays.
Ready to send
Waiting ACK
Got ACK Send Data
LS
Ready to receive
Done
Read DataSend ACK
HSMinimize information flow from HS to LS.
Enforce reasonable performances, i.e.: <average ACK delay as seen from LS> = <average HS ACK delay>
NRL Pump (Probabilistic) Properties:
10
Covert Channel Experimental Results (1)Pdec(h): probability of making a decision within h time units.Pwrong(h): probability of making the wrong decision within h time unitsWe can compute the probability of making the right decision within h time units as: Pright(h) = Pdec(h)(1 - Pwrong(h)). Of course we want Pright(h) to be small.We studied the previous probabilities for various settings of our model parameters.BUFFER SIZE 3 5WINDOW SIZE 3 5OBS WINDOW SIZE 3 5About 2 days of computation for each setting on a 2GHz Intel Pentium PC with Linux OS and 512MB of RAM).
11
Covert Channel Exp: Pdec, Pwrong as a function of the number of steps h
12
Covert Channel Exp: Pright as a Function of the number of steps h
Our time unit is about the time needed to transfer messages from/to the pump (about 1ms).
Our experimental results show that the high system can send bits to the low system at a rate of about 1 bit every 10 seconds, i.e. 0.1 bits/sec.
This is secure enough for many applications.
13
Reliability Analysis:Probabilistic Model Checking (1)
Sometimes we can associate a probability with each transition. In such cases reachability analysis becomes the task of computing the stationary distribution of a Markov Chain. This can be done using a Probabilistic Model Checker (state space too big for matrices).
0
1
2
0.4
0.6
0.3
0.7
0.8
0.2
14
A Control System
Controller Gas Turbine(Turbogas)
Disturbances: electric users, param. var, etc
Vrot: Turbine Rotation speedTexh: Exhaust smokes TemperaturePel: Generated Electric PowerPmc: Compressor Pressure
Settings Fuel Valve OpeningFG102
Vrot, Texh, Pel, Pmc
15
User Demand Distribution
Where: M = max user demand (MAX_U), a = speed of variation of user demand (MAX_D_U)
0.4 + b*(v – M)*|v – M| /M2 when i = 1p(v, i) = 0.2 when i = 0 0.4 + b*(M - v)*|M - v| /M2 when i = -1-0.4 <= b <= 0.4
Let u(t) be the user demand at time t. We can define the (stochastic) dynamics of the user demand as follows:
min(u(t) + a, M) with probability p(u(t), 1)u(t + 1) = u(t) with probability p(u(t), 0) max(u(t) - a, 0) with probability p(u(t), -1)
The further u(t) from u0 (nominal user demand) the higher u(t) probability to return towards u0. That is to decrease when u(t) > u0, to increase when u(t) < u0.
16
Finite Horizon Markov Chain Analysis… of our turbogas
MAX_D_U(KW/sec)
Visited States
RulesFired
Horizon CPU time (s) Probability of violating spec
2500 3,018,970 8,971,839 1600 68562 7.373292e-05
3500 2,226,036 6,602,763 1400 50263 1.076644e-04
4500 1,834,684 5,439,327 1300 41403 9.957147e-05
5000 83,189 246,285 900 2212 3.984375e-03
17
Mutual Exclusion (Mutex)n1 t1
c1 S2=t
2 & T
=1
S2 =
n2
S1 n2 t2
c2 S1=t
1 & T
=2
S1 =
n1
S2
1 2T
S1=n1 & S2=t2
S2=n2 & S1=t1
n1, n2, 1 t1, n2, 1 c1, n2, 1 n1, t2, 1 t1, t2, 1 c1, t2, 1
n1, c2, 1 t1, c2, 1 c1, c2, 1 n1, n2, 2 t1, n2, 2 c1, n2, 2
n1, t2, 2 t1, t2, 2 c1, t2, 2 n1, c2, 2 t1, c2, 2 c1, c2, 2
SPECMutual exclusion: AG (S1 != c1 | S2 != c2) … trueNo starvation S1: AG (S1 = t1 --> AF (S1 = c1)) … true
18
Mutex (~ arbitrary initial state)
Mutual exclusion: AG (S1 != c1 | S2 != c2) …No starvation S1: AG (S1 = t1 --> AF (S1 = c1)) …
n1 t1
c1 S2=t
2 & T
=1
S2 =
n2S1 n2 t2
c2 S1=t
1 & T
=2
S1 =
n1
S2
1 2T
S1=n1 & S2=t2
S2=n2 & S1=t1
19
SMV output (mutex)-- specification AG (S1 != c1 | S2 != c2) is true
-- specification AG (S1 = t1 -> AF S1 = c1) is true
resources used:user time: 0.02 s, system time: 0.04 sBDD nodes allocated: 635Bytes allocated: 1245184BDD nodes representing transition relation: 31 + 6
20
Research Activity
• Automatic Verification and Validation (Model Checking) of Hardwware systems: SMV, VIS, BMC.
• Automatic Verification and Validation of Protocols and Software Systems: Murphi, SPIN.
• Automatic Requirements Validation (via Model Checking)• Automatic Verification of Hybrid Systems: Murphi, Hytech.• Automatic Verification of Probabilistic Systems (Reliability Analysis): PRISM,
FHP-Murphi.• Covert Channel Analysis, Security Analysis: FHP-Murphi• Automatic Synthesis of Optimal (Supervisory) Controllers for Finite State Systems.• Optimization of Complex Systems (via Mixed Integer Linear Programming and
SAT)• Planning (via Model Checking).• Datamining
Algorithms and Tools for
Core Algorithms: Graph Exploration, SAT, PLI, OBDDS
21
Research Activity (2)
• WIP: Automatic Synthesis of Controllers for PCM systems (e.g. DC-DC converters, digital amplifiers, etc.).
• WIP: Automatic Design and Verification of Autonomous Systems.
Algorithms and Tools for
22
TOOLS http://www.dsi.uniroma1.it/~tronci/cached.murphi.html
FHP-MurphiFHP-Murphi allows Finite Horizon Analysis of Markov Chains modelling stochastic hybrids systems. Unlike PRISM, FHP-Murphi can handle real numbers.
Caching Murphi (Cmurphi) (also http://www.stanford.edu/)Cmurphi (Rome “La Sapienza”, L’Aquila) is a disk based extension of the Stanford Murphi Verifier. Cmurphi has been used with success by INTEL to verify Cache Coherence Protocols that, because of state explosion, was not possible to verify using Murphi.
23
Automatic Verification: A Money Saver Testing without automation tends to discover errors towards the end of the design flow. Error fixing is very expensive at that point and may delay product release. Methods to discover errors as soon as possible are needed.
0
5
10
15
20
25
30
35
40
1X 3-6X 10X 15-40X 30-70X 40-1000X
Without AutomatedtestingWith Automated testing
Early development Implementation
Number of times more expensive to fix
Err
ors c
a ug h
t (pe
r ce n
t )
Source: Mercury Interactive, Siebel Siemens
24
Open Source Model Checkers
SMV, NuSMV (Carnegie Mellon University, IRST) [smv,VHDL / CTL]
SPIN (Bell Labs) [PROMELA (C like)/ LTL]
Murphi (Stanford, Roma “La Sapienza”, L’Aquila) [Pascal like/assert() style]
VIS (Berkeley, Stanford, Colorado University) [BLIF, Verilog/CTL, LTL]
PVS (Stanford) [PVS/PVS]
TVLA (Tel-Aviv) [TVLA/TVLA]
Java PathFinder (NASA) [Java Bytecode/LTL]
BLAST (Berkeley) [C/assert()]
Here are a few examples of open source model checkers.
25
Java Verification (BANDERA)SAnToS Group at Kansas State University
26
Some Commercial Model Checkers
Cadence (Verilog, VHDL)
Synopsis (Verilog, VHDL)
Innologic (Verilog)
Telelogic (inside SDL suite)
Esterel
Coverity (C, C++)
Here are a few examples of commercial model checkers.
27
In House Model Checkers
FORTE (INTEL) [Verilog, VHDL/Temporal Logic]
SLAM (Microsoft) [C/assert()]
BEBOP (Microsoft) [C/assert()]
Rule Based (IBM) [Verilog, VHDL/CTL, LTL]
CANVAS (IBM) [Java/constraints-guarantees]
Verisoft (Bell Labs) [C/C]
Here are a few examples of in house model checkers.
28
Summing Up
Automatic Verification (reachability analysis) is a very useful tool for design and analysis of complex systems such as: digital hardware, embedded software and hybrid systems.
Decrease the probability of leaving undetected bugs in our design, thus increasing design quality.
Speed up the testing/simulation process, thus decreasing costs and time-to-market.
Early error detection, thus decreasing design costs.
Support exploration of more complex, hopefully more efficient, solutions by supporting their debugging.
Automatic Verification allows us to: