Carnegie Mellon University

A View from the Engine Room: A View from the Engine Room: Computational Support for Computational Support for Symbolic Model Checking Symbolic Model Checking

A View from the Engine Room: A View from the Engine Room: Computational Support for Computational Support for Symbolic Model Checking Symbolic Model Checking

http://www.cs.cmu.edu/~bryant

Randal E. Bryant

– 2 –25MC

OutlineOutline

Boolean Reasoning as Engine for Model CheckingBoolean Reasoning as Engine for Model Checking BDDs & SAT

An Evaluation of SATAn Evaluation of SAT Current capabilities & limitations Making further progress

Beyond SATBeyond SAT Enhancing DPLL to do more than find single solution

– 3 –25MC

The Origins of Symbolic Model CheckingThe Origins of Symbolic Model Checking

1987 notes by Ken McMillan

Backward traversal of Petri net state space

Realized that reachability could be performed via symbolic Boolean manipulation

– 4 –25MC

Role of Boolean Manipulation in MCRole of Boolean Manipulation in MC

Contributions of BDDs to Model CheckingContributions of BDDs to Model Checking Separate problem from implementation

BDDs provide clean API to model checker

Performed well for many examples

The Emergence of SATThe Emergence of SAT Initially for bounded model checking [Biere, et al., ’96] More recently for full model checking

SAT enumeration [McMillan ’02] Interpolation-based abstraction-refinement [McMillan ’03]

Important PointImportant Point Advances in Boolean manipulation drive progress in model

checking

– 5 –25MC

Recent Progress in SAT SolvingRecent Progress in SAT Solving

766

147 118 81 46

3600

0

1,000

2,000

3,000

Gra

sp (2

000)

zChaf

f (200

1)

BerkM

in (2

002)

zChaf

f (200

3-04

)

Siege

(200

4)

SatElit

eGTI (

2005

)

Ru

n-t

ime

(sec

.)

– 6 –25MC

Conventional Wisdom on SATConventional Wisdom on SAT

BDDs vs. DPLLBDDs vs. DPLL DPLL better than BDDs for straight SAT

Especially problems with large numbers of variables

Best Research Strategy is to Keep Refining DPLL Best Research Strategy is to Keep Refining DPLL Certainly has lead to big improvements!

ClaimClaim This wisdom is overly simplistic

– 7 –25MC

Comparing Parity TreesComparing Parity Trees

Compare linear chain of XORs to randomly trees

Known hard problem for resolution-based SAT solvers

16 n-input trees for different values of n

– 8 –25MC

Parity: Exhaustive TestingParity: Exhaustive Testing

Testing 109 cases is no big deal

Exhaustive

0

100

200

300

400

500

600

700

800

900

0 8 16 24 32 40 48 56

Bits

CP

U s

ec

s.

OK

TIME

– 9 –25MC

Parity: DPLL (ca. 2002 Limmat)Parity: DPLL (ca. 2002 Limmat)

Known difficult problem for DPLL

Limmat

0

100

200

300

400

500

600

700

800

900

0 8 16 24 32 40 48 56

Bits

CP

U s

ec

s.

OK

TIME

– 10 –25MC

Parity: DPLL (MiniSAT)Parity: DPLL (MiniSAT)

Recent SAT solvers have made remarkable progress

MiniSAT

0

100

200

300

400

500

600

700

800

900

0 8 16 24 32 40 48 56

Bits

CP

U s

ec

s.

OK

TIME

– 11 –25MC

Parity: BDDsParity: BDDs

Trivial problem for BDDs

BDD

0

100

200

300

400

500

600

700

800

900

0 8 16 24 32 40 48 56

Bits

CP

U s

ec

s.

OK

TIME

– 12 –25MC

Associativity TestingAssociativity Testing

Typical of arithmetic verification problems Evaluate for different argument word sizes

int addL (int x, int y, int z){ return (x+y)+z;}

int addR(int x, int y) (int x, int y, int z){ return x+(y+z);}

?

=

int mulL (int x, int y, int z){ return (x*y)*z;}

int mulR(int x, int y) (int x, int y, int z){ return x*(y*z);}

?

=

– 13 –25MC

Associativity of AdditionAssociativity of Addition

Easy for BDDs Recent DPLL handle readily

Add Associativity

0

100

200

300

400

500

600

700

800

900

0 8 16 24 32Word Size

MiniSAT

BDD

Exhaustive

– 14 –25MC

Associativity of MultiplicationAssociativity of Multiplication

BDDs better than DPLL

Mult Associativity

0

100

200

300

400

500

600

700

800

900

0 2 4 6 8 10 12Word Size

MiniSAT

BDD

– 15 –25MC

Associativity of MultiplicationAssociativity of Multiplication

Both worse than exhaustive

Mult Associativity

0

100

200

300

400

500

600

700

800

900

0 2 4 6 8 10 12Word Size

MiniSAT

BDD

Exhaustive

– 16 –25MC

Progress in SAT ResearchProgress in SAT Research

Evolution of DPLLEvolution of DPLL Incremental advances yielding more than incremental

improvements Encourages continued incrementing

DownsideDownside Gene pool of SAT solvers diminishing All use DPLL, nonchronological backtracking, 2-literal

watching … New approaches must overcome high performance standard

ClaimClaim We need to be looking beyond incremental changes

– 17 –25MC

Breaking FreeBreaking Free

Raise the Bar on BenchmarksRaise the Bar on Benchmarks Identify challenge benchmarks Examples

Arithmetic problemsBreaking cryptosystems or secure hashesCombinatorial optimization

Parameterize to allow scaling analysis

Acknowledge Value of Niche SolversAcknowledge Value of Niche Solvers Don’t worry about problems that current solvers handle well

– 18 –25MC

BDD/DPLL HybridsBDD/DPLL Hybrids

Very Different ApproachesVery Different Approaches DPLL: Search for one solution from top down BDDs: Encode all solutions from bottom up

Significant Recent EffortSignificant Recent Effort BDD preprocessing for SAT solver [Jin & Somenzi, ’04] DPLL on ZDD-represented clause sets [Aloul, et al., ’01] Satisfy conjunction of BDDs [Damiano & Kukula, ’03, Franco

et al., ’04]

EvaluationEvaluation Incomplete Can help when one approach (BDD / DPLL) much better than

other But what about problems that neither does well?

– 19 –25MC

Beyond SATBeyond SAT

Dealing With QuantifiersDealing With Quantifiers DPLL as QBF solver has had limited success Strength for BDDs

Especially with deep, alternating quantifier nestingE.g., model checking

UnsatisfiabilityUnsatisfiability Impressive progress on generating proofs and unsat cores

Using scaffolding from DPLL

Many applicationsE.g., refinement steps in model checking

No counterpart with BDDs

– 20 –25MC

Challenge Problem: Quantifier EliminationChallenge Problem: Quantifier Elimination

Core Problem For Model CheckingCore Problem For Model Checking Bit-level: Relational product Predicate abstraction

Flanagan & Qadeer, ’02, Lahiri, Bryant, Cook, ’03

MethodsMethods BDDs: quantifier elimination

Use early quantification

DPLL: SAT enumerationPlaisted, ’00, Gupta, et al., ’00, McMillan ’02, Clarke et al., ’03

F

...

...

X

YG...Y

G = X F

– 21 –25MC

Quantifier Elimination ExampleQuantifier Elimination Example

Example from Predicate AbstractionExample from Predicate Abstraction Lahiri, Bryant, Cook, ’03 G = X F Current state variables X Next state variables Y

[ (xx1 xx2 xx3 xx4 xx5 xx6)

(xx1 xx2 xx3 xx4 xx5 xx6) ]

CurrentState

xx11, , xx22, , xx33 , , xx44, , xx55, , xx66

TransitionConstraints

(xx2 yy2) (yy2 yy1) (xx4 xx6 yy1)

xx3 yy4 xx4 yy3

xx5 yy6 xx6 yy5

– 22 –25MC

1 0 1 0 1 0 0 1 0 1 0 1

1 0 1 0 1 0 1 0 0 1 0 1

1 0 0 1 0 1 1 0 1 0 1 0

x1 x2 x3 x4 x5 x6 y1 y2 y3 y4 y5 y6

1 0 1 0 1 0 0 0 0 1 0 1

Set EnumerationSet Enumeration Run SAT checker over formula Generate blocking clause for each newly generated

element

(y1 y2 y3 y4 y5 y6)

[ (xx1 xx2 xx3 xx4 xx5 xx6)

(xx1 xx2 xx3 xx4 xx5 xx6) ] (xx2 yy2) (yy2 yy1) (xx4 xx6 yy1)

xx3 yy4 xx4 yy3

xx5 yy6 xx6 yy5

– 23 –25MC

y1 y2 y3 y4 y5 y6

0 0 0 1 0 1

0 1 0 1 0 1

1 0 0 1 0 1

1 0 1 0 1 0

Compressing Set RepresentationCompressing Set Representation

Disjunct set elements to form BDD Extract prime implicants from BDD Experience: 10X reduction in number of terms

BDDRep.

y1 y2 y3 y4 y5 y6

0 * 0 1 0 1

* 0 0 1 0 1

1 0 1 0 1 0

– 24 –25MC

SAT Enumeration ObservationsSAT Enumeration Observations

PerformancePerformance Better than BDDs when |X| >> |Y| Only have to enumerate for unique assignments to Y

ImprovementsImprovements Attempt to enlarge solution as enumerate [McMillan ’02] Build into DPLL search loop

Lahiri, Nieuwenhuis, Oliveras, ’06Handle successful cases similarly to failures

Make solver stop before it assigns values to all variables Implemented?

ObservationObservation Enumerative methods seem inelegant

– 25 –25MC

ConclusionsConclusions

25MC = 20OBDD25MC = 20OBDD Boolean methods have driven much of the progress in

model checkingBDDs & SAT

SAT ProgressSAT Progress Impressive, but still room for improvement

Beyond SATBeyond SAT Quantifiers Unsatisfiability

Comments?Comments?