Evidence-Based Verification Li Tan Computer Science Department Stony Brook Joint work with Rance...
-
Upload
elfrieda-ryan -
Category
Documents
-
view
216 -
download
2
Transcript of Evidence-Based Verification Li Tan Computer Science Department Stony Brook Joint work with Rance...
Evidence-Based Verification
Evidence-Based Verification
Li Tan
Computer Science DepartmentStony Brook
Joint work with Rance Cleaveland
Augest 2002
Evidence-Based Verification
OutlinePart I. Evidence-based Verification.
1. Motivations.2. The general framework. 3. Applications.
Part II. Evidence-based Model Checking.1. Introducing support set as checker-independent
evidence.2. Extracting support set from existing checkers.3. Post-model-checking analysis based on support sets.
1. Efficiently certifying verification result.2. Generating the diagnostic information.3. Evaluating the quality of model-checking process.
4. Prototype work on the Concurrency Workbench (CWB-NC).
Evidence-Based Verification
Automatic Verification Verification algorithm (checker)
decides in a fully automatic fashion whether or not a transition system satisfies a property.
A simple "Yes/No" may not satisfy users.
Why does my design go wrong [CGMZ95]?
Could my design satisfy the property trivially [KV99]?
Can I trust the verification result [Nam01]?
Evidence-Based Verification
Understanding the verification resultTo answer these questions, users may demand, 1. Diagnostic information. A diag. routine usually reuse the
proof already computed by a checker, 1. Implementation requires the understanding of
checkers.2. Migrating a diag. routine onto a different checker
requires changes on both diag. routine and checker.
3. Proof used for one diagnostic schema may not be suitable for a different schema.
2. Measurement on how well a system has been checked.1. Currently we use “trial and error” strategy to find
out unchecked subformula.3. Evidence to support verification result. Currently we lack
of the proof of correctness which is,1. Independent of the checker, and2. Able to be verified efficiently.
Evidence-Based Verification
Evidence-Based Verification
Checker 1 Checker n
Verifier
Diagnostic schema 1..k
Invalid Proof
Checker 2
Certification of result
Evaluating verification process
…
…
Let the result carry its own certifiable and check-independent proof
Portable Proof of Correctness
Evidence-Based Verification
The general framework Defining abstract proof structures (APS).
APS encodes the proof structures of different checkers in a standard form.
APS may be used as the certification for correctness of result.
APS is rich enough to support a variety of analyses, while still abstract enough to save the space.
APS can be verified independently and efficiently. Extracting APS from existing checkers.
Extraction should NOT compromise the complexities of checkers.
Utilizing support set to perform diagnoses. Certifying verification result. Generating diagnostic information. Measuring the quality of verification process
Evidence-Based Verification
Part II. Evidence-based Model Checking:An introduction by case study
Evidence-Based Verification
Boolean Equation System=Temporal Property+Transition System
Evidence-Based Verification
Support Set
Evidence-Based Verification
Support Set
Evidence-Based Verification
Support Sets (Continue)Support set reflects how a checker “reasons”
model-checking problem. By properties 1 and 2, support set implies
a fixpoint solution for BES. By property 3, support set respects the
semantics of fixpoint operators in BES. Theorem 1 [TanCle02]
There exists a support set =<r, X,> , [E](X)=r.
Evidence-Based Verification
Support sets for other temporal logics
Boolean equation system (BES)=transition system
+ temporal property. Model checkers explicitly or implicitly construct BES
. Variables in BES stands for pairs of subformula and
state in transition system. Decorated support set <, T, >, where =<r, X,
>, resolves subformulas and states associated with the variables in In our example,
T(X0)= s0 …… (X0)= AG(a ) AF b) ……
Evidence-Based Verification
Extracting Support SetThe extraction is, practical. Support sets can be extracted
from a wide range of existing checkers, Boolean-Graph algorithm [And92], Linear
Alternation-Free algorithms[CleSte91], On-the-fly algorithms for full -calculus LAFP [LRS98] and SLP [TanCle02b], Automaton-based model checkers([BhaCle96a] and [KVW00]).
efficient. The overhead doesn't affect the original complexities of these checkers.
simply. We only need to record the immediate dependency of variables.
Evidence-Based Verification
Application I: Certifying model-checking results
Checking (a) and (b) can be done in linear time.
Checking (c) can be reduced to even-loop problem (a O(n log ad) problem[KKV01]).
Model checking is a NP Å co-NP problem [EmeJutSis93].
The cost of certifying results < The cost of model checking.
Evidence-Based Verification
Application II: Generating Counterexample1. Reducing a support set to a linear support set,
Support Set hr, X, i is linear if |(Xi)| · 1 for every Xi defined on .
Evidence-Based Verification
Application II: Generating Counterexample (Cont.)
A counterexample can be generated by, “Projecting” linear support set on states
Removing the redundant steps, hs, X’i should be removed if …hs, Xi, hs, X’i is
not interleaved with a modal operator.
Evidence-Based Verification
Application III:Evaluate the quality of MC
A positive result may hide the problem
T may pass AG(c ) AF b) trivially because c never occurs in T.
Is there the status of a state (Minicoverage [CKV01]) or a subformula (Vacuity [KV99]) irrelevant to the result?
Coverage problem of support set. Has support set covered all the states
and properties?
Evidence-Based Verification
Evaluate the quality of Model-checking process (Cont.)
1. The support set for s0 ² AG(c ) AF b) is like,
2. AF b is not covered ) AF b is not checked.
Evidence-Based Verification
Furture Work I:A Client-Server Model for model checking
Server: checkers Inputting system and properties encoded in
some temporal logic. Outputting support set.
Client: user interface, diagnostic generation, and evidence-verifier.
Design Systems and Properties
Abstract Proof Structures
Evidence-Based Verification
Future Work II:Proof-Carrying Code
Mobile code [Nec97] carries its own proof attesting to its safeness.
Currently compilers are modified to produce the proof for a predefined set of safety rules.
Integrate support-set-ready model checkers with compilers.
Certifying compiler enjoy the richness of temporal logics.
Evidence-Based Verification
A Prototype on CWB-NC
Evidence-Based Verification
ConclusionCheckers produce abstract proof structures as evidence. Extracting APS won't affect the complexities
of checkers. APS provides the portable evidence for
justifying verification result. Applications of APS.
Efficiently certifying the verification result. Evaluating the quality of verification. Generating a wide range of diagnostic information.
APSs are defined for Model checking, Equiv. checking, and Preordering Checking.
Evidence-Based Verification
A Prototype on CWB-NC
Evidence-Based Verification
ConclusionCheckers produce support sets as evidence.
Support set is independent of checker. Extracting support sets won't affect the
complexities of checkers. Support set justifies the correctness of result. Support set attests to the quality of
verification. A wide range of diagnostic information can be
built on support set. Linear Counterexample and witness. Synthesizing winning strategy for model-checking
game. Vacuity Detection and Coverage Metrics.