SAT Based Abstraction/Refinement in Model-Checking

Based on work by

E. Clarke, A. Gupta, J. Kukula, O. Strichman (CAV’02)

2

Model Checking

Given a: Finite transition system M(S, I, R, L) A temporal property p

The model checking problem: Does M satisfy p?

3

Model Checking

Temporal properties: “Always x=y”

(G(x=y)) “Every Send is followed immediately

by Ack” (G(Send X Ack))

“Reset can always be reached” (GF Reset)

“From some point on, always switch_on” (FG switch_on)

“Safety” properties

“Liveness” properties

4

Model Checking (safety)

I

Add reachable states until reaching a fixed-point

= bad state

5

Model Checking (safety)

I

Too many states to handle != bad state

6

Abstraction

h h hh h

Abstraction Function h : S ! S’

S

S’

7

Abstraction Function

Partition variables into visible(V) and invisible(I) variables.

The abstract model consists of V variables. I variables are made inputs.

The abstraction function maps each state to its projection over V.

8

Abstraction Function

0 0

0 0 0 00 0 0 10 0 1 00 0 1 1

h

x1 x2 x3 x4

x1 x2

Group concrete states with identical visible part to a single abstract state.

10

Existential Abstraction

I

I

11

Model Checking Abstract Model

Preservation Theorem

The counterexample may be spurious

Converse does not hold

12

Checking the Counterexample

Counterexample : (c1, …,cm) Each ci is an assignment to V.

Simulate the counterexample on the concrete model.

13

Checking the Counterexample

Concrete traces corresponding to the counterexample:

(Initial State)

(Unrolled Transition Relation)

(Restriction of V to Counterexample)

14

Abstraction-Refinement Loop

Check Counterexample

Refine

Model CheckAbstract

M’, pM, p, hNo Bug

Pass

Fail

BugRealSpurious

h’

15

Refinement methods…

PFrontier

Inputs

Invisible

Visible

(R. Kurshan, 80’s)Localization

16

Generate all counterexamples.Prioritize variables according to their consistency in the counterexamples.

X1 x2 x3 x4

(Glusman et al., 2002) Intel’s refinement heuristic

Refinement methods…

17

Simulate counterexample on concrete model with SAT

If the instance is unsatisfiable, analyze conflict

Make visible one of the variables in the clauses that

lead to the conflict

(Chauhan, Clarke, Kukula, Sapra, Veith, Wang, FMCAD 2002) Abstraction/refinement with conflict analysis

Refinement methods…

18

Why spurious counterexample?

I

I

Deadend states

Bad States

Failure State

f

19

Refinement

Problem: Deadend and Bad States are in the same abstract state. Solution: Refine abstraction function.The sets of Deadend and Bad states should be separated into different abstract states.

20

Refinement

h’

h’

h’

h’

h’

Refinement : h’

h’

h’

21

RefinementDeadend States

22

RefinementDeadend States

Bad States

23

Refinement as Separation

0 1 0 1 0 1 0

0 0 1 0 0 1 0

0 1 1 1 0 1 0

d1

b1

b2

I

V

0

1

1

1

0

1

Refinement : Find subset U of I that separates between all pairs of deadend and bad states. Make them visible.

Keep U small !

24

Refinement as Separation

0 1 0 1 0 1 0

0 0 1 0 0 1 0

0 1 1 1 0 1 0

d1

b1

b2

0

1

1

I

V

Refinement : Find subset U of I that separates between all pairs of deadend and bad states. Make them visible.

Keep U small !

25

Refinement as Separation

The state separation problemInput: Sets D, BOutput: Minimal U I s.t.: d D, b B, u U. d(u) b(u)

The refinement h’ is obtained by adding U to V.

26

Two separation methods

ILP-based separation Minimal separating set. Computationally expensive.

Decision Tree Learning based separation. Not optimal. Polynomial.

27

Separation with Decision Tree learning (Example)

Separating Set : {v1,v2,v4}

D B

B D BD

10 0 1

b1d2d1b2

v1

v4v2

0 1{d1,b2} {d2,b1}

DB

Classification:

28

Separation with 0-1 ILP (Example)

29

Separation with 0-1 ILP

One constraint per pair of states. vi = 1 iff vi is in the separating set.

30

Refinement as Learning

For systems of realistic size Not possible to generate D and B. Expensive to separate D and B.

Solution: Sample D and B Infer separating variables from the samples.

The method is still complete: counterexample will eventually be eliminated.

31

Efficient Sampling

D Bd b

Let (D,B) be the smallest separating set of D and B.

Q: Can we find it without deriving D and B ?

A: Search for smallest d,b such that (d,b) = (D,B)

32

Efficient Sampling

Direct search towards samples that contain more information.

How? Find samples not separated by the current separating set (Sep).

33

Efficient Sampling

Recall: D characterizes the deadend states B characterizes the bad states D B is unsatisfiable

Samples that agree on the sep variables:

Rename all vi B to

vi’

34

Efficient Sampling

Sep = {}d,b = {}

Run SAT solveron (Sep)

STOPunsat

Compute Sep:= (d,b)

Add samples to d and b

sat

Sep is the minimal separating set of D and B

35

The Tool

NuSMV CadenceSMV

MC

Chaff

SAT

LpSolve

Dec Tree

Sep

36

ResultsProperty 1

37

ResultsProperty 2

Efficient Sampling together with Decision Tree

Learning performs best.

Machine Learning techniques are useful in

computing good refinements.