Quantitative Timed Analysis of Interactive Markov Chains€¦ · Interactive Markov Chains Dennis...

Quantitative Timed Analysis ofInteractive Markov Chains

Dennis Guck1 Tingting Han2

Joost-Pieter Katoen1 Martin R. Neuhaußer3

1 RWTH Aachen University, Germany

2 University of Oxford, UK

3 Saarland University, Germany

4th NASA Formal Methods SymposiumApril 4, 2012

Dennis Guck (RWTH) Quantitative Timed Analysis of IMCs April 4, 2012 1 / 23

Introduction Stochastic models

Why stochastic models?

Random delays.

External effects (like system inputs).

Concurrent behavior of components.

Reliability of components.

Availability of components.

expected time and long-run average


Introduction Stochastic models

Stochastic models

discrete-time continuous-time

deterministic DTMC CTMC

non-deterministicMDP CTMDP

IMC


Introduction Why to use Interactive Markov Chains?

Why to use Interactive Markov Chains?

Model-based performance evaluation

Analyse performance metrics based on abstract system model

The prevailing paradigm is continuous-time randomness

Complexity of systems requires compositional approach

Enormous model sizes require compositional abstraction mechanisms

Nondeterminism is at heart of compositionality


Introduction Why to use Interactive Markov Chains?

Use of Interactive Markov Chains

Interactive Markov Chains (IMCs) are used for compositional semantics of:

Architecture Analysis & Design Language (AADL)

Dynamic fault trees (DFT)

Generalized stochastic petri nets (GSPN)

Scenario-aware dataflow languages

Globally asynchronous locally synchronous hardware (GALS)

Stochastic statecharts

The foundation of IMCs are fundamental for these.


Introduction What are Interactive Markov Chains?

What are Interactive Markov Chains?

Interactive Markov chain

An interactive Markov chain is a tuple I = (S ,Act, 99K, −→ , s0) where:

S is a nonempty, finite set of states with initial state s0 ∈ S ,

Act is a finite set of actions,

99K ⊆ S × Act × S is a set of action transitions and

−→ ⊆ S × R>0 × S is a set of Markovian transitions.

s0

s1

s2

s3

s4

6

44

2

α

β

α

1


Introduction What are Interactive Markov Chains?

Maximal progress

s0

s1

s2

τλ

reduces to s2

s0

s1

τ

Note: The IMC is not subject to any further synchronisation.


Introduction An Example

Dynamic Fault Tree

x

FDEP

A B

PAND

Note: There is only the minimum and the maximum expected time due to nondeterminism.



Dynamic Fault Tree

x

FDEP

A B

PAND

A?

B?

B? F !




Dynamic Fault Tree

x

FDEP

A B

PAND

A?

B?

B? F !

x?

A?

A!




Dynamic Fault Tree

x

FDEP

A B

PAND

A?

B?

B? F !

x?

A?

A!

λ α!




Dynamic Fault Tree

x

FDEP

A B

PAND

fail

λ1 λ2 λ3

λ4 D!

E !

F




Dynamic Fault Tree

x

FDEP

A B

PAND

fail

λ1 λ2 λ3

λ4 D!

E !

F

What is the expected time to fail?



Expected time Definition

Expected time

The elapsed time on a path before visiting some goal state in G ⊆ S .

Minimal expected time

eTmin(s,♦G ) = infD

Es,D(VG ) = infD

∫Paths

VG (π) Prs,D

(dπ).

minimum time t on an infinite path π, such that G ∩ π@t 6= ∅

Note: Only the amount of time before entering a goal state in G is relevant.


Expected time Definition

Expected time

Function for minimal expected time

[L(v)] (s) =

1

E (s)+

∑s′∈S

P(s, s ′) · v(s ′) if s ∈ MS \ G

mins α−−→ s′

v(s ′) if s ∈ IS \ G

0 if s ∈ G .

Corollary

eTmin(s,♦G ) is the unique fixpoint of L.


Expected time Example

Example

s0

s1

s2

s3

s4

6

44

2

α

β

α

1

DetermineeTmin(si ,♦{s4})

G = {s4}⇒ eTmin(s4,♦G ) = 0 = v(s4)

Maximize: v(s0) + v(s1) + v(s2) + v(s3)Subject to:

v(s0) ≤ 110 + 3

5v(s1) + 25v(s2)

v(s1) ≤ 16 + 2

3v(s2) + 13v(s3)

v(s2) ≤ v(s1)

v(s2) ≤ v(s4)

v(s3) ≤ v(s0)



Example

s0

s1

s2

s3

s4

6

44

2

α

β

α

1


G = {s4}⇒ eTmin(s4,♦G ) = 0 = v(s4)


v(s0) ≤ 110 + 3

5v(s1) + 25v(s2)

v(s1) ≤ 16 + 2

3v(s2) + 13v(s3)

v(s2) ≤ v(s1)

v(s2) ≤ v(s4)

v(s3) = v(s0)



Example

s0

s1

s2

s3

s4

6

44

2

α

β

α

1


G = {s4}⇒ eTmin(s4,♦G ) = 0 = v(s4)


v(s0) ≤ 110 + 3

5v(s1) + 25v(s2)

v(s1) ≤ 16 + 2

3v(s2) + 13v(s3)

v(s2) = min{v(s1), v(s4)}

v(s3) = v(s0)



Example

s0

s1

s2

s3

s4

6

44

2

α

β

α

1


G = {s4}⇒ eTmin(s4,♦G ) = 0 = v(s4)


v(s0) ≤ 110 + 3

5v(s1) + 25v(s2)

v(s1) ≤ 16 + 2

3v(s2) + 13v(s3)

v(s2) = 0

v(s3) = v(s0)



Example

s0

s1

s2

s3

s4

6

44

2

α

β

α

1


G = {s4}⇒ eTmin(s4,♦G ) = 0 = v(s4)


v(s0) ≤ 110 + 3

5v(s1)

v(s1) ≤ 16 + 1

3v(s0)

v(s2) = 0

v(s3) = v(s0)



Example

s0

s1

s2

s3

s4

6

44

2

α

β

α

1


G = {s4}⇒ eTmin(s4,♦G ) = 0 = v(s4)


v(s0) = 14

v(s1) = 14

v(s2) = 0

v(s3) = v(s0)


Long-run average Definition

Long-run average

The amount of time we spent in G ⊆ S in the long-run.

Long-run average

LRAD(s,G ) = Es,D(AG ) =

∫Paths

AG (π) Prs,D

(dπ).

fraction of time spent in G on an infinite path π

Note: We search for the minimum infD or maximum supD long-run average.



Intuition

s0

s1

s2 s8

s3s4s5

s6 s7

s9

s10α2

α1

42

β1

β2

6

3

γ2

γ1

20γ3

5

5 44

κ1κ2

κ3

Determine the maximal end components {I1, . . . Ik} of IMC I.

Determine Lra(G ) in maximal end components Ij .Reduce the computation of Lras0,G to an SSP problem.



Intuition

s0

s1

s2 s8

s3s4s5

s6 s7

s9

s10α2

α1

42

β1

β2

6

3

γ2

γ1

20γ3

5

5 44

κ1κ2

κ3

Determine the maximal end components {I1, . . . Ik} of IMC I.

Determine Lra(G ) in maximal end components Ij .Reduce the computation of Lra(s0,G ) to an SSP problem.



Determine the long-run average

Long-run ratio

R(π) = limn→∞

∑n−1i=0 c1(si , αi )∑n−1j=0 c2(sj , αj)

.

c1(s, σ) =

{1

E(s)if s ∈ MS ∩ G ∧ σ = ⊥

0 otherwise,c2(s, σ) =

{1

E(s)if s ∈ MS ∧ σ = ⊥

0 otherwise.

The ratio between the average residence time for states in G against all states in S .



Determine the long-run average

Minimum long-run ratio for MDP

Rmin(s) = infD

Es,D(R) = infD

∑π∈Paths

R(π) · Prs,D(π).

Theorem

For unichain IMC I, Lramin(s,G ) equals Rmin(s) in MDP M(I).


Long-run average Example

Transform into linear programming problem

Use real variables k for the long-run and xs for each s ∈ S :

Maximize kSubject to:

xs 6 c1(s, α)− k · c2(s, α) +∑s′∈S

P(s, α, s ′) · xs′ .

for each s ∈ S and α ∈ Act.



Example

s0

s1

s2 s8

s3s4s5

s6 s7

s9

s10α2

α1

42

β1

β2

6

3

γ2

γ1

20γ3

5

5 44

κ1κ2

κ3

I1 with state space S1 = {s4, s5, s6, s7}Lramin

1 (G ) = 23

I2 with state space S1 = {s2, s3, s8, s9, s10}Lramin

2 (G ) = 913

Solving SSP problem:

Lramin(s0,G ) = 80117

G = {s7, s8}DetermineLramin(s0,G )



Example

s0

s1

s2 s8

s3s4s5

s6 s7

s9

s10α2

α1

42

β1

β2

6

3

γ2

γ1

20γ3

5

5 44

κ1κ2

κ3


1 (G ) = 23


2 (G ) = 913

Solving SSP problem:Lramin(s0,G ) = 80

117


Maximize: kSubject to:

xs4 ≤ xs5

xs4 ≤ xs7

xs5 ≤ − 120k + xs7

xs6 ≤ xs5

xs7 ≤ 110− 1

10k + 1

2xs4 + 1

2xs4



Example

s0

s1

s2 s8

s3s4s5

s6 s7

s9

s10α2

α1

42

β1

β2

6

3

γ2

γ1

20γ3

5

5 44

κ1κ2

κ3


1 (G ) = 23


2 (G ) = 913


117




Example

s0

s1

s2 s8

s3s4s5

s6 s7

s9

s10α2

α1

42

β1

β2

6

3

γ2

γ1

20γ3

5

5 44

κ1κ2

κ3


1 (G ) = 23


2 (G ) = 913


117


Maximize: kSubject to:

xs2 ≤ xs3

xs2 ≤ xs8

xs3 ≤ − 19k + 2

3xs8 + 1

3xs9

xs8 ≤ 18− 1

8k + 1

2xs3 + 1

2xs9

xs9 ≤ xs8

xs9 ≤ xs10

xs10 ≤ xs3



Example

s0

s1

s2 s8

s3s4s5

s6 s7

s9

s10α2

α1

42

β1

β2

6

3

γ2

γ1

20γ3

5

5 44

κ1κ2

κ3


1 (G ) = 23


2 (G ) = 913


117




Example

s0

s1

Lramin1 (G) Lramin

2 (G)

α2

α1

42


1 (G ) = 23


2 (G ) = 913


117




Example

s0

s1

Lramin1 (G) Lramin

2 (G)

α2

α1

42


1 (G ) = 23


2 (G ) = 913


117


Maximize: xs0 + xs1

Subject to:

xs0 ≤ 913

xs0 ≤ xs1

xs1 ≤ 13· 2

3+ 2

3· 9

13



Example

s0

s1

Lramin1 (G) Lramin

2 (G)

α2

α1

42


1 (G ) = 23


2 (G ) = 913


117



Case Studies Workstation cluster

Workstation cluster (Haverkort, Hermanns & Katoen(2000))

GSPN of the workstation cluster as described in (Haverkort, Hermanns & Katoen(2000)).


Case Studies Google file system

Google file system (Cloth and Haverkort(2005))

GSPN of the Google file system as described in (Cloth and Haverkort(2005)).


Case Studies IMCA

Interactive Markov Chain Analyzer (IMCA)

http://www-i2.informatik.rwth-aachen.de/imca/


Case Studies Results

Case studies (Workstation cluster)

eTmax(s,♦G) Prmax(s,♦G) Lramax(s, G)N # states # transitions |G | time (s) time (s) time (s)1 111 320 74 0.0009 0.0061 0.00464 819 2996 347 0.0547 0.0305 0.11378 2771 10708 1019 0.6803 0.3911 1.3341

16 10131 40340 3419 10.1439 5.3423 20.027832 38675 156436 12443 292.7389 94.0289 455.438752 100275 408116 31643 3187.1171 1807.7994 OOM

Computation times for the workstation cluster.


Case Studies Results

Case studies (Google file system)

eTmin(s,♦G) Prmin(s,♦G) Lramin(s, G)M # states # transitions |G | time (s) time (s) time (s)10 1796 6544 408 1.6568 0.1584 0.141120 7176 27586 1713 2.6724 2.5669 14.980430 16156 63356 3918 11.3836 14.2459 35.065440 28736 113928 7023 31.1416 48.8603 236.530860 64696 202106 15933 142.2179 315.8246 OOM

Computation times for Google file system (S = 5000 and N = 100000).


Conclusion

Conclusion

What we have seen

Novel algorithms for expected time and long-run average

Prototypical tool support in IMCA (http://www-i2.informatik.rwth-aachen.de/imca/)

Both objectives can be reduced to SSP problems

What we investigate

Generalization to Markov automata

combination of continous time and probabilistic transitions

Experimentation with symbolic data structures such as BDDs

e.g. by exploiting PRISM for MDP analysis


Quantitative Timed Analysis of Interactive Markov Chains€¦ · Interactive Markov Chains Dennis...

Documents

Transcript of Quantitative Timed Analysis of Interactive Markov Chains€¦ · Interactive Markov Chains Dennis...