Property-Directedk-Induction · Introduction boundedmodelchecking Canfindbugs,cannotproveproperties...

Property-Directed k-Induction

Dejan Jovanović Bruno Dutertre

SRI International

FMCAD 2016, Mountain View, CA

Thanks to NASA

Outline

1 Introduction

2 Property-Directed k-Induction

3 Experimental Evaluation

Introductionthe problem

Given a transition systemS = ⟨I, T⟩withx⃗: state variables,I(⃗x): initial state formula,T(⃗x, x⃗′): state transition formula,

check whether all reachable states satisfy a property P.

Example: ZenoGivenS = ⟨I, T⟩with

I ≡ (x = 0) ∧ (y = 0.5) , T ≡ (x′ = x+ y) ∧ (y′ = y/2) ,

check whether (x < 1).

Introductionthe problem

Given a transition systemS = ⟨I, T⟩withx⃗: state variables,I(⃗x): initial state formula,T(⃗x, x⃗′): state transition formula,

check whether all reachable states satisfy a property P.

Example: ZenoGivenS = ⟨I, T⟩with

I ≡ (x = 0) ∧ (y = 0.5) , T ≡ (x′ = x+ y) ∧ (y′ = y/2) ,

check whether (x < 1).

Automation goals1 Find bugs2 Prove properties

Introductionboundedmodel checking

Can find bugs, can not prove propertiesCan use off-the-shelf SAT/SMT solverFor non-trivial systems unrolling can be expensive

Finite reachabilityX


I(⃗x0) ∧ ¬P(⃗x0)




I(⃗x0) ∧ T(⃗x0, x⃗1) ∧ ¬P(⃗x1)




I(⃗x0) ∧ T(⃗x0, x⃗1) ∧ T(⃗x1, x⃗2) ∧ ¬P(⃗x2)




I(⃗x0) ∧ T(⃗x0, x⃗1) ∧ T(⃗x1, x⃗2) ∧ T(⃗x2, x⃗3) ∧ ¬P(⃗x3)




I(⃗x0) ∧ T(⃗x0, x⃗1) ∧ T(⃗x1, x⃗2) ∧ T(⃗x2, x⃗3) ∧ ¬P(⃗x3)

Can find bugs, can not prove propertiesCan use off-the-shelf SAT/SMT solverFor non-trivial systems unrolling can be expensiveFinite reachabilityX

Introductioninduction

I(⃗x0)⇒ P(⃗x0)

P(⃗x0) ∧ T(⃗x0, x⃗1)⇒ P(⃗x1)

Can prove propertiesCan use off-the-shelf SAT/SMT solver


I(⃗x0)⇒ P(⃗x0)

P(⃗x0) ∧ T(⃗x0, x⃗1)⇒ P(⃗x1)


P

I


I(⃗x0)⇒ P(⃗x0)

P(⃗x0) ∧ T(⃗x0, x⃗1)⇒ P(⃗x1)


P P

T


I(⃗x0)⇒ P(⃗x0)

P(⃗x0) ∧ T(⃗x0, x⃗1)⇒ P(⃗x1)


P P P

T T


I(⃗x0)⇒ P(⃗x0)

P(⃗x0) ∧ T(⃗x0, x⃗1)⇒ P(⃗x1)


P P P P

T T T


I(⃗x0)⇒ P(⃗x0)

P(⃗x0) ∧ T(⃗x0, x⃗1)⇒ P(⃗x1)



I(⃗x0)⇒ P(⃗x0)

P(⃗x0) ∧ T(⃗x0, x⃗1)⇒ P(⃗x1)

Can prove propertiesCan use off-the-shelf SAT/SMT solverZeno: property (x < 1) is not inductive

ZenoI ≡ (x = 0) ∧ (y = 0.5)

T ≡ (x′ = x+ y) ∧ (y′ = y/2)P ≡ (x < 1)


I(⃗x0)⇒ P(⃗x0)

P(⃗x0) ∧ T(⃗x0, x⃗1)⇒ P(⃗x1)

Can prove propertiesCan use off-the-shelf SAT/SMT solverZeno: property (x < 1) ∧ (x+ 2y ≤ 1) is inductive

ZenoI ≡ (x = 0) ∧ (y = 0.5)

T ≡ (x′ = x+ y) ∧ (y′ = y/2)P ≡ (x < 1)

Introductionk-induction

I(⃗x0)⇒ P(⃗x0)I(⃗x0) ∧ T(⃗x0, x⃗1)⇒ P(⃗x1)I(⃗x0) ∧ T(⃗x0, x⃗1) ∧ T(⃗x1, x⃗2)⇒ P(⃗x2)P(⃗x0) ∧ T(⃗x0, x⃗1) ∧ P(⃗x1) ∧ T(⃗x1, x⃗2) ∧ P(⃗x2) ∧ T(⃗x2, x⃗3)⇒ P(⃗x3)

Can find bugs, can prove propertiesCan use off-the-shelf SAT/SMT solverFor non-trivial systems unrolling can be expensive


I(⃗x0)⇒ P(⃗x0)

I(⃗x0) ∧ T(⃗x0, x⃗1)⇒ P(⃗x1)I(⃗x0) ∧ T(⃗x0, x⃗1) ∧ T(⃗x1, x⃗2)⇒ P(⃗x2)P(⃗x0) ∧ T(⃗x0, x⃗1) ∧ P(⃗x1) ∧ T(⃗x1, x⃗2) ∧ P(⃗x2) ∧ T(⃗x2, x⃗3)⇒ P(⃗x3)


P

I


I(⃗x0)⇒ P(⃗x0)I(⃗x0) ∧ T(⃗x0, x⃗1)⇒ P(⃗x1)

I(⃗x0) ∧ T(⃗x0, x⃗1) ∧ T(⃗x1, x⃗2)⇒ P(⃗x2)P(⃗x0) ∧ T(⃗x0, x⃗1) ∧ P(⃗x1) ∧ T(⃗x1, x⃗2) ∧ P(⃗x2) ∧ T(⃗x2, x⃗3)⇒ P(⃗x3)


P P

TI


I(⃗x0)⇒ P(⃗x0)I(⃗x0) ∧ T(⃗x0, x⃗1)⇒ P(⃗x1)I(⃗x0) ∧ T(⃗x0, x⃗1) ∧ T(⃗x1, x⃗2)⇒ P(⃗x2)

P(⃗x0) ∧ T(⃗x0, x⃗1) ∧ P(⃗x1) ∧ T(⃗x1, x⃗2) ∧ P(⃗x2) ∧ T(⃗x2, x⃗3)⇒ P(⃗x3)


P P P

T TI




P P P

T T

P

T




P P P

T T

P

T

P

T



Can find bugs, can prove propertiesCan use off-the-shelf SAT/SMT solverFor non-trivial systems unrolling can be expensiveExample: property (|x| < 1) is not inductive

Stronger

I ≡ (x = 0) ∧ (y = 0)

T ≡ (x′ =3

5x+

2

5y) ∧ (|y′| < 1)

P ≡ (|x| < 1)



Can find bugs, can prove propertiesCan use off-the-shelf SAT/SMT solverFor non-trivial systems unrolling can be expensiveExample: property (|x| < 1) is 2-inductive

Stronger

I ≡ (x = 0) ∧ (y = 0)

T ≡ (x′ =3

5x+

2

5y) ∧ (|y′| < 1)

P ≡ (|x| < 1)

Introductionstrengthening

Key problem: find a strengthening that proves the property

I(⃗x0)⇒ P(⃗x0)P(⃗x0) ∧ T(⃗x0, x⃗1)⇒ P(⃗x1)

F (⃗x)⇒ P(⃗x)

T

P P

Same for k-inductionIs k-induction stronger?X



I(⃗x0)⇒ F (⃗x0)F (⃗x0) ∧ T(⃗x0, x⃗1)⇒ F (⃗x1)F (⃗x)⇒ P(⃗x)

T

P

L1

FP

L1

F





T

P

L1

L2

FP

L2

L1

F





T

P

L1

L2

L3

FP

L2

L3

L1

F





T

P

L1

L2

L3

FP

L2

L3

L1

FP

L1

L2

L3

F

Same for k-induction

Is k-induction stronger?X




T

P

L1

L2

L3

FP

L2

L3

L1

FP

L1

L2

L3

F


Introductiontimeline

InductionBoundedmodel checking [BCCZ99]k-induction [SSS00]Interpolation-basedmodel checking [McM03]IC3/PDR [Bra11]

based on inductionincremental strengtheningno unrolling: lots of “easy” queriesinterpolation-based learning

Lots of work on SMT-based extensions [HB12, CG12, KGC14, CGMT14]

Outline

1 Introduction



Property-Directed k-Inductionmodules

SMT solvingmore than SAT/UNSAT

1-step reachabilitymore than reachable/unreachable

k-step reachabilitymore than reachable/unreachable

k-inductionsearch for a strengthening and learn from failures

Property-Directed k-Induction1-step reachability

R

FT

Basic satisfiability query

R(⃗x) ∧ T(⃗x, x⃗′) ∧ F(⃗x′)

SAT

: generalize the counterexample to G YICES2 with [KGC14]

UNSAT: interpolate, with J refuting F MATHSAT5


R

FT


R(⃗x) ∧ T(⃗x, x⃗′) ∧ F(⃗x′)

SAT

: generalize the counterexample to G YICES2 with [KGC14]UNSAT: interpolate, with J refuting F MATHSAT5


R

FTG


R(⃗x) ∧ T(⃗x, x⃗′) ∧ F(⃗x′)

SAT : generalize the counterexample to G YICES2 with [KGC14]

UNSAT: interpolate, with J refuting F MATHSAT5


R

FT

J


R(⃗x) ∧ T(⃗x, x⃗′) ∧ F(⃗x′)

SAT : generalize the counterexample to G YICES2 with [KGC14]UNSAT: interpolate, with J refuting F MATHSAT5

Property-Directed k-Inductionk-step reachability

Reachability in k stepsGiven F that is not reachable in< k steps, check if it’s reachable in k steps.

R0 R1 R2

T T T

Ri valid up to i

1-step backward searchlearn and refineRi

all the way: reachableunreachable: learnlearned fact valid up to k



R0 R1 R2

T T T

Ri valid up to i1-step backward search

learn and refineRi




R0 R1 R2

T T T

Ri valid up to i1-step backward searchlearn and refineRi




R0 R1 R2

T T T


all the way: reachable

unreachable: learnlearned fact valid up to k



R0 R1 R2

T T T



Property-Directed k-Inductionmain procedure

Require: S = ⟨I, T⟩ and I⇒ P1 function PD-KIND(S, P)2 n← 03 F ← {(P,¬P)}4 loop5 pick k-induction depth 1 ≤ k ≤ n+ 16 ⟨F ,G, np⟩ ← PUSH(S,F , P, n, k)7 if Pmarked invalid then return invalid8 if F = G then return valid9 n← np10 F ← G

Setup

single reasoning frameFreasoning index nobligations (FABS, FCEX) ∈ FFABS is valid up to nFCEX ¬P, FABS refutes FCEX

InitiallyP is valid up to n = 0

¬P ¬P, P refutes¬P



Setupsingle reasoning frameF

reasoning index nobligations (FABS, FCEX) ∈ FFABS is valid up to nFCEX ¬P, FABS refutes FCEX





Setupsingle reasoning frameFreasoning index n

obligations (FABS, FCEX) ∈ FFABS is valid up to nFCEX ¬P, FABS refutes FCEX





Setupsingle reasoning frameFreasoning index nobligations (FABS, FCEX) ∈ F

FABS is valid up to nFCEX ¬P, FABS refutes FCEX





Setupsingle reasoning frameFreasoning index nobligations (FABS, FCEX) ∈ FFABS is valid up to n

FCEX ¬P, FABS refutes FCEX





Setupsingle reasoning frameFreasoning index nobligations (FABS, FCEX) ∈ FFABS is valid up to nFCEX ¬P, FABS refutes FCEX




F

F F

valid in frames 0, ..., n

induction check

T F F F


F

F F

valid in frames 0, ..., n

k-induction check

T F T F ... T F T F F F


Fvalid in frames 0, ..., n n+1, ..., npF


Fvalid in frames 0, ..., n Gn+1, ..., npF

Property-Directed k-inductionthe PUSH procedure

Pick an obligation (FABS, FCEX) ∈ F

F ∧ T ∧ . . . ∧ F ∧ T⇒ FABSF ∧ T ∧ . . . ∧ F ∧ T ∧ FCEX



F ∧ T ∧ . . . ∧ F ∧ T⇒ FABS

F ∧ T ∧ . . . ∧ F ∧ T ∧ FCEX

Is FABS k-inductive relative toF?



F ∧ T ∧ . . . ∧ F ∧ T⇒ FABS

F ∧ T ∧ . . . ∧ F ∧ T ∧ FCEX

Is FABS k-inductive relative toF? If yes, push itX



F ∧ T ∧ . . . ∧ F ∧ T⇒ FABS

F ∧ T ∧ . . . ∧ F ∧ T ∧ FCEX

Is FABS k-inductive relative toF? If no, get the generalization GCTI of the CTI



F ∧ T ∧ . . . ∧ F ∧ T⇒ FABS

F ∧ T ∧ . . . ∧ F ∧ T ∧ FCEX

Can we get to FCEX?

If yes, then generalize to GCEX

If GCEX reachable, then we have a counter-example to PXIf GCEX not reachable, learn lemma to eliminate GCEXX



F ∧ T ∧ . . . ∧ F ∧ T⇒ FABS

F ∧ T ∧ . . . ∧ F ∧ T ∧ FCEX

Can we get to FCEX? If yes, then generalize to GCEX

If GCEX reachable, then we have a counter-example to PXIf GCEX not reachable, learn lemma to eliminate GCEXX




We have a generalization GCTI of the CTI, and can not get to FCEX

If GCTI reachable,weaken FABS to¬FCEXXIf GCTI not reachable, learn lemma and strengthen FABSX




We have a generalization GCTI of the CTI, and can not get to FCEXIf GCTI reachable,weaken FABS to¬FCEXXIf GCTI not reachable, learn lemma and strengthen FABSX

Outline

1 Introduction



Experimental Evaluationoverall

Z3 SPACER NUXMV PD-KINDproblem set X ⊤/⊥ time X ⊤/⊥ time X ⊤/⊥ time X ⊤/⊥ timeapproximate-agreement (9) 9 8/1 213 7 6/1 1150 9 8/1 2174 9 8/1 164azadmanesh-kieckhafer (20) 20 17/3 3404 20 17/3 4678 20 17/3 294 20 17/3 192cav12 (99) 69 48/21 2102 71 49/22 3529 72 50/22 7443 71 49/22 4990conc (6) 4 4/0 128 4 4/0 655 6 6/0 421 4 4/0 270ctigar (110) 64 44/20 1683 72 52/20 4249 76 56/20 1342 77 57/20 2823hacms (5) 1 1/0 11 1 1/0 4 4 3/1 388 5 3/2 1661lustre (790) 757 421/336 1888 763 427/336 2263 760 424/336 7660 774 438/336 3494oral-messages (9) 9 7/2 16 9 7/2 44 9 7/2 161 9 7/2 2tta-startup (3) 1 1/0 9 1 1/0 8 1 1/0 17 1 1/0 8tte-synchro (6) 6 3/3 969 6 3/3 445 5 2/3 405 6 3/3 21unified-approx (11) 8 5/3 2928 11 8/3 589 11 8/3 139 11 8/3 217

948 559/389 13351 965 575/390 17614 973 582/391 20444 987 595/392 13842

timeout of 20 minutes, Z3 [HB12], NUXMV [CGMT14], SPACER [KGC14]

Experimental Evaluationas a variant of IC3/PDR

Z3 SPACER NUXMV PD-KIND∞ PD-KIND1problem set X ⊤/⊥ time X ⊤/⊥ time X ⊤/⊥ time X ⊤/⊥ time X ⊤/⊥ timeapproximate-agreement (9) 9 8/1 213 7 6/1 1150 9 8/1 2174 9 8/1 164 9 8/1 155azadmanesh-kieckhafer (20) 20 17/3 3404 20 17/3 4678 20 17/3 294 20 17/3 192 20 17/3 107cav12 (99) 69 48/21 2102 71 49/22 3529 72 50/22 7443 71 49/22 4990 74 50/24 6404conc (6) 4 4/0 128 4 4/0 655 6 6/0 421 4 4/0 270 5 5/0 164ctigar (110) 64 44/20 1683 72 52/20 4249 76 56/20 1342 77 57/20 2823 73 53/20 4920hacms (5) 1 1/0 11 1 1/0 4 4 3/1 388 5 3/2 1661 1 1/0 2lustre (790) 757 421/336 1888 763 427/336 2263 760 424/336 7660 774 438/336 3494 769 431/338 2019oral-messages (9) 9 7/2 16 9 7/2 44 9 7/2 161 9 7/2 2 9 7/2 74tta-startup (3) 1 1/0 9 1 1/0 8 1 1/0 17 1 1/0 8 2 1/1 742tte-synchro (6) 6 3/3 969 6 3/3 445 5 2/3 405 6 3/3 21 6 3/3 60unified-approx (11) 8 5/3 2928 11 8/3 589 11 8/3 139 11 8/3 217 11 8/3 158

948 559/389 13351 965 575/390 17614 973 582/391 20444 987 595/392 13842 979 584/395 14805

timeout of 20 minutes, Z3 [HB12], NUXMV [CGMT14], SPACER [KGC14]

Experimental Evaluationoverall

Effective and robust on real-world problemsGood at both proving properties and finding bugsk-induction: can prove properties using a smaller strengtheningk-induction: the only engine that can prove all k-inductive propertiesk-induction: effective bug-finder due to the longer steps of k-induction

Experimental Evaluationk-induction

Summary

Newmethod for infinite-state systems:variant of IC3/PDR based on k-inductioneffective in practice: proofs and bugsfocuses on induction rather than bugsno SMT query left behindmore powerful than k-inductionmodular: tunable, amenable to heuristicsimplemented in SALLY (fork me at GitHub)

http://sri-csl.github.io/sally/

References I

[BCCZ99] Armin Biere, Alessandro Cimatti, Edmund Clarke, and Yunshan Zhu.Symbolic model checking without BDDs.Tools and Algorithms for the Construction and Analysis of Systems, pages 193–207, 1999.

[Bra11] Aaron R Bradley.SAT-basedmodel checking without unrolling.In Verification, Model Checking, and Abstract Interpretation, pages 70–87, 2011.

[CG12] Alessandro Cimatti and Alberto Griggio.Software model checking via IC3.In Computer Aided Verification, pages 277–293, 2012.

[CGMT14] Alessandro Cimatti, Alberto Griggio, Sergio Mover, and Stefano Tonetta.IC3 modulo theories via implicit predicate abstraction.In Tools and Algorithms for the Construction and Analysis of Systems, pages 46–61. 2014.

[HB12] Kryštof Hoder and Nikolaj Bjørner.Generalized property directed reachability.In Theory and Applications of Satisfiability Testing, pages 157–171. 2012.

[KGC14] Anvesh Komuravelli, Arie Gurfinkel, and Sagar Chaki.SMT-basedmodel checking for recursive programs.In Computer Aided Verification, pages 17–34, 2014.

References II

[McM03] Kenneth L McMillan.Interpolation and SAT-basedmodel checking.In International Conference on Computer Aided Verification, pages 1–13, 2003.

[SSS00] Mary Sheeran, Satnam Singh, and Gunnar Stålmarck.Checking safety properties using induction and a SAT-solver.In Formal Methods in Computer-Aided Design, pages 127–144, 2000.

Property-Directedk-Induction · Introduction boundedmodelchecking Canfindbugs,cannotproveproperties...

Documents

Transcript of Property-Directedk-Induction · Introduction boundedmodelchecking Canfindbugs,cannotproveproperties...