Abstraction for Falsification
description
Transcript of Abstraction for Falsification
![Page 1: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/1.jpg)
Abstraction for FalsificationAbstraction for Falsification
Thomas Ball
Orna Kupferman
Greta Yorsh
Microsoft Research, Redmond, US
Hebrew University, Jerusalem, Israel
Tel Aviv University, Israel
CAV’05
![Page 2: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/2.jpg)
Abstraction for VerificationAbstraction for Verification
• Goal: prove properties
• Sound abstraction for verification– properties of abstract system hold for
corresponding concrete system
: C A– if abstract state a satisfies property P then all
concrete states represented by a satisfy P
![Page 3: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/3.jpg)
Abstraction for VerificationAbstraction for Verification
• Goal: prove properties
• Sound abstraction for verification– properties of abstract system hold for
corresponding concrete system
: C A a A if a P
then c C . (c)=a c P
![Page 4: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/4.jpg)
Abstraction for VerificationAbstraction for Verification
• Goal: prove properties
• Sound abstraction for verification– properties of abstract system hold for
corresponding concrete system
: C A a A if a P
then c C . (c)=a c P
FalsificationFalsification
detect errors
![Page 5: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/5.jpg)
Abstraction for VerificationAbstraction for Verification
• Goal: prove properties
• Sound abstraction for verification– errors of the abstract system exist in
corresponding concrete system
: C A a A if a P
then c C . (c)=a c P
FalsificationFalsification
falsificationdetect errors
![Page 6: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/6.jpg)
Abstraction for VerificationAbstraction for Verification
• Goal: prove properties
• Sound abstraction for verification– errors of the abstract system exist in
corresponding concrete system
: C A a A if a P
then c C . (c)=a c P
FalsificationFalsification
falsificationdetect errors
c C . (c)=a c P
![Page 7: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/7.jpg)
MotivationMotivation
• An abstraction that is sound for falsification need not be sound for verification.
• Existing frameworks for abstraction for verification – Modal Transition System (MTS)– MTS, PKS,KMTS - equivalent in expressive
power [ Godefroid,Jagadessan – VMCAI’03 ]
– can be too restrictive for falsification
![Page 8: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/8.jpg)
Main ResultsMain Results
• New framework for abstraction – Ternary Modal Transition System (TMTS)– TMTS is stronger than MTS– Semantics of -calculus for TMTS
• Weak reachability– TMTS with parameterized transitions gives
tighter underapproximation– TMTS with assume-guarantee transitions for
complete reasoning
![Page 9: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/9.jpg)
may
Modal Transition SystemsModal Transition Systems
underapproximation
overapproximation
Concrete Abstract
a
a’
total
a
a’
must
c. (c) = a c’ . (c’) = a’ c c’
MAY(a,a’)MAY(a,a’)
MUST+(a,a’)MUST+(a,a’)
MUSTMUST––(a,a’)(a,a’)
c, c’ . c c’ (c) = a (c’) = a’
(existential abstraction)
must may
underapproximation
c’. (c’) = a’ c. (c) = a c c’onto
a
a’
must
[ T. Ball - FMCO’04 ]
must maymust+ and must– are incomparable
![Page 10: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/10.jpg)
TMTS strictly more expressive than MTSTMTS strictly more expressive than MTS
MTS • may and must+ transitions• precision preorder is logically characterized by PML
::= p | AX | |
TMTS• may, must+ and must– transitions• precision preorder is logically characterized by full-PML
::= p | AX | AY | |
• full-PML is strictly more expressive than PML [Pinter,Wolper - PODC’84] [Kupferman,Pnueli - LICS’95]
![Page 11: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/11.jpg)
TMTS: what does it buy us?TMTS: what does it buy us?
• Verifying specifications with past operators
• Reasoning about specifications in falsification setting– must+ for verification and must- for falsification
• Tighter weak reachability in abstract system – combine must+ and must- along the path
![Page 12: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/12.jpg)
Semantics of Semantics of -calculus for TMTS-calculus for TMTS
: C A• (C, c1)
• [ (A, a1) ] - the value of the -calculus formula in state a1 of TMTS A
![Page 13: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/13.jpg)
• [ (A, a) ] = T – for all concrete state c with (c) = a, (C, c)
• [ (A, a) ] = T – there exists a concrete state c with (c) = a and (C, c)
• [ (A, a) ] = F– for all concrete state c with (c) = a, (C, c)
• [ (A, a) ] = F
– there exists a concrete state c with (c) = a and (C, c) • [ (A, a) ] = M
– there exist concrete states c and c’ such that
(c) = (c’) = a and (C, c) and (C, c’) • [ (A, a) ] =
Semantics of Semantics of -calculus for TMTS-calculus for TMTS
![Page 14: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/14.jpg)
Information Information LatticeLattice
T F
Truth Truth LatticeLattice
T
F
![Page 15: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/15.jpg)
Information Information LatticeLattice
T F
Truth Truth LatticeLattice
T F
M
T
F
F
T
M
![Page 16: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/16.jpg)
Semantics of Semantics of -calculus for TMTS-calculus for TMTS
• [ (A, a) 1 2 ]
• [ (A, a) EX ]• [ (A, a) ]
![Page 17: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/17.jpg)
[ (A, a) 1 2 ] =
[ (A, a) 1 ] # [ (A, a) 2 ]
6-valued Semantics of 6-valued Semantics of 11 22
![Page 18: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/18.jpg)
# F F M T T
F F F F F F F
F F F F F F F
M F F ? F M F
T F F F ? T
T F F M ? T
F F F
6-valued Semantics of 6-valued Semantics of 11 22
![Page 19: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/19.jpg)
# F F M T T
F F F F F F F
F F F F F F F
M F F ? F M F
T F F F ? T
T F F M T T
F F F
6-valued Semantics of 6-valued Semantics of 11 22
![Page 20: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/20.jpg)
# F F M T T
F F F F F F F
F F F F F F F
M F F ? F M F
T F F F T
T F F M T T
F F F
6-valued Semantics of 6-valued Semantics of 11 22
![Page 21: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/21.jpg)
Information Information LatticeLattice
T F
Truth Truth LatticeLattice
T F
M
T
F
F
T
M
![Page 22: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/22.jpg)
# F F M T T
F F F F F F F
F F F F F F F
M F F ? F M F
T F F F T
T F F M T T
F F F
6-valued Semantics of 6-valued Semantics of 11 22
![Page 23: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/23.jpg)
# F F M T T
F F F F F F F
F F F F F F F
M F F F F M F
T F F F T
T F F M T T
F F F
6-valued Semantics of 6-valued Semantics of 11 22
![Page 24: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/24.jpg)
[ (A, a) EX ] =
Semantics of EXSemantics of EX
F if for all a’, if may(a,a’) then [(A, a’) ] = F
T if exists a’ s.t. must+(a,a’) and [(A,a’) ] = T
T if exists a’ s.t. must–(a,a’) and [(A,a’) ] T
otherwise
![Page 25: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/25.jpg)
c’
a EX = T
a’
must–
= T
c
• [ (A, a) EX ] = T
• exists a’ s.t. must–(a,a’) and [(A,a’) ] = T
• exists c’ such that (c’)=a’ and c’ • for all c’ with (c’)=a’ there is c
with (c)=a such that cc’
if [ (A, a) EX ] = T then there exists c with (c) = a and c EX
EX
![Page 26: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/26.jpg)
Semantics of Semantics of
• The semantics of PML operators is monotonic
– Least fixpoint operator can be computed by iterations from F is the usual way:
– [(A,a) Z . (Z) ] = [ (A, a) *(F) ]
![Page 27: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/27.jpg)
• The 6-valued semantics is at least as precise as the standard 3-valued semantics of -calculus for MTS
• [(A,a) ] = – 3-valued abstraction refinement of must+ transitions
[Shoham,Grumberg – CAV’03] adapt for must-
• Hypermust transitions– [Larsen,Xinxin-LICS‘90] [Shoham,Grumberg – CAV’04]– adapt for must– – MTS with hypermust+ is incomparable with TMTS
EX(x>6) T EX(x>6) F EX(x>6) = T
Semantics of Semantics of -calculus for TMTS-calculus for TMTS
EX(x>6) = ?
must –
x = 7x = 10
may
x > 6
x > 6
x:=x–3
7 8 9 ...
7 8 9 ...
![Page 28: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/28.jpg)
Semantics of Semantics of -calculus for TMTS-calculus for TMTS
• The 6-valued semantics is at least as precise as the standard 3-valued semantics of -calculus for MTS
• [(A,a) ] = – 3-valued abstraction refinement of must+ transitions
[Shoham,Grumberg – CAV’03] adapt for must-
• Hypermust transitions– [Larsen,Xinxin-LICS‘90] [Shoham,Grumberg – CAV’04]– adapt for must– – MTS with hypermust+ is incomparable with TMTS
![Page 29: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/29.jpg)
Weak ReachabilityWeak Reachability
• a’ is weakly-reachable from a c, c’ . (c)=a (c’)=a’ c * c’
c
c’ a’
ainitial state
error state
error trace
Related to testing
![Page 30: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/30.jpg)
L1: TF L0: FT L0: FF
L2: TF L3: FT L2: FF
L4: FT L4: FFL4: TF
x<6 x>7 (x=6)(x=7)
may
may must– must–
must–must–
L0: if x<6 then
L1: x:= x + 3
L2: if x > 7 then
L3: x :=x – 3
L4:
Predicates: (x < 6) (x > 7)
ExampleExample
![Page 31: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/31.jpg)
L1: TF L0: FT L0: FF
L2: TF L3: FT L2: FF
L4: FT L4: FFL4: TF
x<6 x>7 (x=6)(x=7)
may
may must– must–
must–must–
L0: if x<6 then
L1: x:= x + 3
L2: if x > 7 then
L3: x :=x – 3
L4:
Predicates: (x < 6) (x > 7)
ExampleExample
x = 5
![Page 32: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/32.jpg)
Underapproximation of Underapproximation of Weak ReachabilityWeak Reachability
• if [must+]*(a,a’) then a’ is weakly reachable from a
• Arbitrary combinations of must+ and must– transitions do not preserve weak reachability
• Find a tighter underapproximation of weak-reachability
![Page 33: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/33.jpg)
L1: TF L0: FT L0: FF
L2: TF L3: FT L2: FF
L4: FT L4: FFL4: TF
x<6 x>7 (x=6)(x=7)
may
may must– must–
must–must–
L0: if x<6 then
L1: x:= x + 3
L2: if x > 7 then
L3: x :=x – 3
L4:
Predicates: (x < 6) (x > 7)
ExampleExample
must – ?must + ?
x = 9
x = 6
x = 5
x = 2
![Page 34: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/34.jpg)
Underapproximation of Underapproximation of Weak ReachabilityWeak Reachability
• if [must+]*(a,a’) then a’ is weakly reachable from a
• Arbitrary combinations of must+ and must– transitions do not preserve weak reachability
• Find a tighter underapproximation of weak-reachability
![Page 35: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/35.jpg)
ObservationsObservations
• a3 is weakly reachable from a1
if there exists a2 such that
must–(a1,a2) and must+(a2,a3)
• Onto nature of must– is preserved by [must-]*
• Total nature of must+ is preserved by [must+]*
a3
must+
a1
a2
must–
[T.Ball – FMCO’04]
![Page 36: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/36.jpg)
UnderapproximationUnderapproximation
If there exists a1, a2, a3 such that
[must–]*(a1,a2) and
[must+]*(a2,a3)
then a3 is weakly-reachable from a1
a3
[must+]*
a1
a2
[must–]*
[T.Ball – FMCO’04]
![Page 37: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/37.jpg)
L1: TF L0: FT L0: FF
L2: TF L3: FT L2: FF
L4: FT L4: FFL4: TF
x<6 x>7 (x=6)(x=7)
may
may must– must–
must–must–
L0: if x<6 then
L1: x:= x + 3
L2: if x > 7 then
L3: x :=x – 3
L4:
Predicates: (x < 6) (x > 7)
ExampleExample
![Page 38: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/38.jpg)
a
a’
( total from a? )MUST+ ?MUST+ ?
( onto a’ ?)MUSTMUST– – ??
NONO
NONO
MAYMAY
Parameterized TransitionsParameterized Transitions
![Page 39: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/39.jpg)
a
a’
must+()
total from
c. (c) = a c c’ . (c’) = a’ c c’
MUST+(MUST+())
Parameterized TransitionsParameterized Transitions
a
a’
must–()
MUSTMUST–(–())
c’. (c’) = a’ c’ c. (c) = a c c’
onto
if is TRUE then must+() is must+ and must–() is must–
![Page 40: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/40.jpg)
ObservationObservation
• a3 is weakly reachable from a1
if there exists a2 such that
– must–(1)(a1,a2)
– must+(2) (a2,a3)
– 1 2 a2 is satisfiable
a3
must+(2)
a1
a2
must–(1)
12
![Page 41: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/41.jpg)
ObservationObservation
• a3 is weakly reachable from a1
if there exists a2 such that
– must–(1)(a1,a2)
– must+(2) (a2,a3)
– 1 2 a2 is satisfiable
• Strongest parameters 1 and 2
a3
a1
a2
must–(1)
12
must+(2)
![Page 42: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/42.jpg)
a
a’
s
MUST+ ( WP(s,a’) )MUST+ ( WP(s,a’) )
Strongest ParametersStrongest Parameters
Generated automatically as part of the construction of TMTS
c. (c) = a c c’ . (c’) = a’ c c’
if must+() then a ( WP(s,a’))
a
a’
s
MUSTMUST– – ( SP (s,a) )( SP (s,a) )
c’. (c’) = a’ c’ c. (c) = a c c’
if must–() then a ( SP(s,a))
![Page 43: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/43.jpg)
L1: TF L0: FT L0: FF
L2: TF L3: FT L2: FF
L4: FT L4: FFL4: TF
x<6 x>7 (x=6)(x=7)
may
may must– must–
must–must–
L0: if x<6 then
L1: x:= x + 3
L2: if x > 7 then
L3: x :=x – 3
L4:
Predicates: (x < 6) (x > 7)
ExampleExample
SP(x:=x+3, x<6) = x < 9
WP(x:=x-3, x<6) = x < 9
![Page 44: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/44.jpg)
L1: TF L0: FT L0: FF
L2: TF L3: FT L2: FF
L4: FT L4: FFL4: TF
x<6 x>7 (x=6)(x=7)
must– must–
must–must–
L0: if x<6 then
L1: x:= x + 3
L2: if x > 7 then
L3: x :=x – 3
L4:
Predicates: (x < 6) (x > 7)
ExampleExample
SP(x:=x+3, x<6) = x < 9
WP(x:=x-3, x<6) = x < 9
must–(x<9)
must+(x<9)
must– (x < 9)
must+ (x < 9)
![Page 45: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/45.jpg)
Tighter UnderapproximationTighter Underapproximation
If there exists a1,...,a5 s.t.
[must–]*(a1,a2)
must–(1)(a2,a3)
must+(2) (a3,a4)
[must+]*(a4,a5)
1 2 a3 is satisfiable
then a5 is weakly-reachable from a1
a4
a2
a3
12
a5
a1
must+(2)
must–(1)
[must+]*
[must–]*
![Page 46: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/46.jpg)
Complete Reasoning Complete Reasoning
– a’ is reachable by a certain sequence of abstract transitions from a
– a’ is weakly-reachable from a
• Assume-guarantee transitions– another type of parameterized transitions:
<> must+ <’>
![Page 47: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/47.jpg)
a
a’
<>must+<‘ > c. (c) = a c
c’ . (c’) = a’ c’ ’ c c’
< < > MUST+ > MUST+ < < ’ ’ >>
Assume-Guarantee TransitionsAssume-Guarantee Transitions
’
Which and ’ predicates do we need?
’
a
a’
c’. (c’) = a’ c’ ’
c . (c) = a c c c’
< < > MUST> MUST–– < < ’ > ’ >
<>must–<‘ >
![Page 48: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/48.jpg)
The idea...The idea...
33
3 3
is satisfiable
a4
a2
a3
a5
a1
s1
s2
s3
s4
<1>must– <2>
<2>must– <3>
1 = a1
2 = SP(s1, 1) a2
3 = SP(s2, 2) a3
<4>must+ < 5>
<3>must+ < 4>
3 = WP(s3,4) a3
4 = WP(s4,5) a4
5 = a5
![Page 49: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/49.jpg)
Assume-guarantee transitionsAssume-guarantee transitions
• Complete Reasoning about Weak Reachability– a’ is reachable by a certain sequence of
assume-guarantee transitions from a– a’ is weakly-reachable from a
• Finding right parameters ~ computing loop invariants
![Page 50: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/50.jpg)
Weak Reachability: SummaryWeak Reachability: Summary
[must–] * [must+]*must–(1) must+(2)
[must–] * [must+]*
• Previous work [T.Ball – FMCO’04]:
• Parameterized transitions
• Assume-guarantee transitions – complete reasoning
![Page 51: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/51.jpg)
ApplicationsApplications
• Falsification of properties in CTL, LTL
• Abstraction-guided test generation– tighter underapproximation of weakly-
reachable states improves coverage of the generated tests
– example of QuickSort’s partition function
![Page 52: Abstraction for Falsification](https://reader035.fdocuments.in/reader035/viewer/2022062309/56814a2e550346895db74d96/html5/thumbnails/52.jpg)
SummarySummary
• Ternary Modal Transition System (TMTS)– onto and total must transitions– full-PML logical characterizes precision
preorder on TMTS
• 6-valued semantics of -calculus for TMTS
• Tighten underapproximation of weak reachability with parameterized transitions– completeness result using assume-guarantee
transitions