SAT Based Abstraction/Refinement in Model-Checking

Joint work with E. Clarke* A. Gupta* J. Kukula**

*Carnegie Mellon University**Synopsys

Ofer Strichman*

2

Model Checking

I

Add reachable states until reaching a fixed-point

3

Model Checking

Too many states to handle !

I

4

Abstraction

h h hh h

Abstraction Function h : S ! S’

S

S’

5

Abstraction Function

Partition variables into visible(V) and invisible(I) variables.

The abstract model consists of V variables. I variables are made inputs.

The abstraction function maps each state to its projection over V.

6

Abstraction Function

0 0

0 0 0 00 0 0 10 0 1 00 0 1 1

h

x1 x2 x3 x4

x1 x2

Group concrete states with identical visible part to a single abstract state.

7

Building Abstract Model

M’ can be computed efficiently if M is in functional form, e.g. sequential circuits.

Abstract

x1 x2 x3 x4 x1 x2

i1 i2 i1 i2 x3 x4

8

Existential Abstraction

I

I

9

Model Checking Abstract Model

Preservation Theorem

The counterexample may be spurious

Converse does not hold

10

Current trends… (1/3)

PFrontier

Inputs

Invisible

Visible

(R. Kurshan, 80’s)Localization

11

Abstraction-Refinement Loop

Check Counterexample

Refine

Model CheckAbstract

M’, pM, p, hNo Bug

Pass

Fail

BugRealSpurious

h’

12

Why spurious counterexample?

I

I

Deadend states

Failure State

Bad States

f

13

Refinement

Problem: Deadend and Bad States are in the same abstract state.

Solution: Refine abstraction function.

The sets of Deadend and Bad states should be separated into different abstract states.

14

Refinement

h’

h’

h’

h’

h’

Refinement : h’

h’

h’

15

Abstraction

Check Counterexample

Refine

Model CheckAbstract

M’, pM, p, hNo Bug

Pass

Fail

BugRealSpurious

h’

16

Checking the Counterexample

Check Counterexample

Refine

Model CheckAbstract

M’, pM, p, hNo Bug

Pass

Fail

BugRealSpurious

h’

17

Checking the Counterexample

Counterexample : (c1, …,cm) Each ci is an assignment to V.

Simulate the counterexample on the concrete model.

18

Checking the Counterexample

Concrete traces corresponding to the counterexample:

(Initial State)

(Unrolled Transition Relation)

(Restriction of V to Counterexample)

19

Refinement

Check Counterexample

Refine

Model CheckAbstract

M’, pM, p, hNo Bug

Pass

Fail

BugRealSpurious

h’

20

RefinementDeadend States

21

RefinementDeadend States

Bad States

22

Refinement as Separation

0 1 0 1 0 1 0

0 0 1 0 0 1 0

0 1 1 1 0 1 0

d1

b1

b2

I

V

0

1

1

1

0

1

23

Refinement as Separation

0 1 0 1 0 1 0

0 0 1 0 0 1 0

0 1 1 1 0 1 0

d1

b1

b2

0

1

1

I

V

Refinement : Find subset U of I that separates between all pairs of deadend and bad states. Make them visible.

Keep U small !

24

Refinement as Separation

The state separation problemInput: Sets D, BOutput: Minimal U I s.t.: d D, b B, u U. d(u) b(u)

The refinement h’ is obtained by adding U to V.

25

Two separation methods

ILP-based separation Minimal separating set. Computationally expensive.

Decision Tree Learning based separation. Not optimal. Polynomial.

26

Separation with ILP (Example)

27

Separation with ILP

One constraint per pair of states. vi = 1 iff vi is in the separating set.

28

Decision Tree learning (Example)

Separating Set : {v1,v2,v4}

D B

B D BD

10 0 1

b1d2d1b2

v1

v4v2

0 1{d1,b2} {d2,b1}

DB

Classification:

29

Decision Tree Learning

Input : Set of examples Each example is an assignment of

values to the attributes. Each example has a classification.

a1

a5a2

c0 c2c1c1

0

0

0 11

1

Output : Decision Tree Each internal node is a test

on an attribute. Each leaf corresponds to a

classification.

30

Separation using Decision Tree Learning

Attributes : Invisible variables I Classifications : ‘D’ and ‘B’ Example Set : Deadend Bad

Separating Set : The variables on the nodes of the decision tree.

31

Refinement as Learning

For systems of realistic size Not possible to generate D and B. Expensive to separate D and B.

Solution: Sample D and B Infer separating variables from the samples.

The method is still complete: counterexample will eventually be eliminated.

32

Efficient Sampling

D Bd b

Let (D,B) be the smallest separating set of D and B.

Q: Can we find it without deriving D and B ?

A: Search for smallest d,b such that (d,b) = (D,B)

33

Efficient Sampling

Direct search towards samples that contain more information.

How? Find samples not separated by the current separating set (Sep).

34

Efficient Sampling

Recall: D characterizes the deadend states B characterizes the bad states D B is unsatisfiable

Samples that agree on the sep variables:

Rename all vi B to

vi’

35

Efficient Sampling

Sep = {}d,b = {}

Run SAT solveron (Sep)

STOPunsat

Compute Sep:= (d,b)

Add samples to d and b

sat

Sep is the minimal separating set of D and B

36

The Tool

NuSMV CadenceSMV

MC

Chaff

SAT

LpSolve

Dec Tree

Sep

37

ResultsProperty 1

38

ResultsProperty 2

Efficient Sampling together with Decision Tree

Learning performs best.

Machine Learning techniques are useful in

computing good refinements.

39

Current trends… (1/3)

PFrontier

Inputs

Invisible

Visible

(Barner, Geist, Gringauze, CAV’02)

Check counterexample incrementally (‘layering’).Find small set of variables in Sf for which it is impossible to find an assignment consistent with the counterexample.

(Originally: R. Kurshan, 80’s)Localization

40

Current trends… (2/3)

Generate all counterexamples.Prioritize variables according to their consistency in the counterexamples.

X1 x2 x3 x4

(Glusman et al., 2002) Intel’s refinement heuristic

41

Current trends… (3/3)

Simulate counterexample on concrete model

with SAT

If the instance is unsatisfiable, analyze conflict

Make visible one of the variables in the clauses

that lead to the conflict

(Chauhan, Clarke, Kukula, Sapra, Veith, Wang, FMCAD 2002) Abstraction/refinement with conflict analysis

42

Current trends… (3/3)

Remove clauses gradually, until instance becomes satisfiable.Choose invisible variables from the removed set.

43

Current trends (3/3)

(Chauhan, Clarke, Kukula, Sapra, Veith, Wang, FMCAD 2002) Abstraction/refinement with conflict analysis

44

Future Work

Currently: Sometimes we find too many equally ‘good’ refinements to choose from.We need more criteria for a good refinement (not just # latches).

Number of gates, number of clauses Distance from property Fan-in degree

45

Future work

Currently we restart with a refined transition relation

T T T

T ‘ T ‘ T ‘

46

Future work

T ’

T ’

A different approach: restart from the previous state.

TT T

An abstraction/refinement backtrack algorithm What intermediate BDD’s should we save ? How can BDDs be altered rather than recomputed ?

47

The End

48

Generating Samples

Execute Sat Solver

satisfiable sample

Add clause negating assignment to I in

failure state

STOP

“enough samples”/

unsatisfiable

Initialize SAT solver with ( or )