Ymer: A Statistical Model Checker Håkan L. S. Younes Carnegie Mellon University.
15
Ymer: A Statistical Model Checker Håkan L. S. Younes Carnegie Mellon University
-
Upload
lewis-carter -
Category
Documents
-
view
213 -
download
1
Transcript of Ymer: A Statistical Model Checker Håkan L. S. Younes Carnegie Mellon University.
Ymer:A Statistical Model Checker
Håkan L. S. YounesCarnegie Mellon University
Younes Ymer: A Statistical Model Checker 2
Probabilistic Model Checking
Given a model M, a state s, and a property , does hold in s for M ? Model: stochastic discrete event system Property: probabilistic temporal logic formula
Example: ≥0.1[ ≤5 full ]
Younes Ymer: A Statistical Model Checker 3
Statistical Solution Method
Use acceptance sampling to verify probabilistic properties Hypothesis: ≥ [] Observation: verify over a sample path
Bounds on probability of verification error Probability of false negative: ≤ Probability of false positive: ≤
Younes Ymer: A Statistical Model Checker 4
Error Bounds
Actual probability of holding
Pro
babi
lity
of e
rror
whe
n ve
rifyi
ng
≥ [
]
p1 p0
Indifference region
2
Younes Ymer: A Statistical Model Checker 5
Ymer at a Glance
Supports time-homogeneous generalized semi-Markov processes
Limited to time-bounded properties Distributed acceptance sampling (even
with sequential acceptance sampling) Purely statistical approach for verifying
nested probabilistic statements
Younes Ymer: A Statistical Model Checker 6
DistributedAcceptance Sampling
MasterAcceptance Sampling
Slavesimulation
Slavesimulation
Slave Masterregister
model & property
observation
observation
done
Younes Ymer: A Statistical Model Checker 7
Avoiding Sample Bias
Process observations as they come in? No, bias against observations that take a long
time to generate (long sample paths) Process observations according to a
predetermined schedule
1 2 1 1 2
1 1 2
Schedule:
Received:
Younes Ymer: A Statistical Model Checker 8
Case Study:Symmetric Polling System
Single server, n polling stations Stations are attended in cyclic order Each station can hold one message State space of size O(n·2n)
Server
…Polling stations
Younes Ymer: A Statistical Model Checker 9
ResultsP
erc
en
t of s
ing
le m
ach
ine 100
Size of state space102 104 106 108 1010 1012 1014
90
80
70
60
50
Machine 1: 733 MHz Pentium III
Machine 2: 500 MHz Pentium III
Younes Ymer: A Statistical Model Checker 10
Nested Probabilistic Statements: Robot Grid World
Probability is at least 0.9 that goal is reached within 100 seconds while periodically communicating ≥0.9[≥0.5[ ≤9 comm] ≤100 goal ]
Younes Ymer: A Statistical Model Checker 11
Statistical Verification ofNested Probabilistic Statements
Cannot verify path formula without some probability of error Probability of false negative: ≤ ′ Probability of false positive: ≤ ′
Observation error
Younes Ymer: A Statistical Model Checker 12
Performance Considerations
Verification error is independent of observation error Pick observation error to minimize effort
The same state may be visited along multiple sample paths Memoize verification results to avoid repeated
effort
Younes Ymer: A Statistical Model Checker 13
Robot Grid World (results)numericalmixedmixedstatisticalstatistical
Veri
fica
tion
tim
e (
seco
nd
s)
10−2
10−1
100
101
102
103
104
Size of state space102 104 106 108 1010 1012
≥0.9[≥0.5[ ≤9 comm] ≤100 goal ]
= 0.025
= 0.05
= = 10−2
Younes Ymer: A Statistical Model Checker 14
Robot Grid World:Effect of Memoization
Sa
mp
le s
ize
101
102
103
statisticalstatistical
statisticalstatistical
Un
iqu
e/v
isite
d s
tate
s
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
102 104 106 102 104 106
Size of state space Size of state space
Younes Ymer: A Statistical Model Checker 15
Availability
Source code is released under GPL http://sweden.autonomy.ri.cmu.edu/ymer/
