The Value 1 Problem for Probabilistic Automata - LIAFAA characterization 20 fA∗ is the space of...
56
The Value 1 Problem for Probabilistic Automata LIAFA Nathana¨ el Fijalkow LIAFA, Universit´ e Denis Diderot - Paris 7, France Institute of Informatics, Warsaw University, Poland [email protected] June 27th, 2014
Transcript of The Value 1 Problem for Probabilistic Automata - LIAFAA characterization 20 fA∗ is the space of...
-
The Value 1 Problem
for Probabilistic Automata
LIAFA
Nathanaël Fijalkow
LIAFA, Université Denis Diderot - Paris 7, France
Institute of Informatics, Warsaw University, Poland
June 27th, 2014
-
1Probabilistic automata (Rabin, 1963)
1
2
3
b
a, 0.4
b
a, 0.6
a, b
a
PA : A∗ → [0, 1]
PA(w) is the probability that a run for w ends up in F
-
2The value 1 problem
This talk is about the value 1 problem:
INPUT: A a probabilistic automatonOUTPUT: for all ε > 0, there exists w ∈ A∗, PA(w) ≥ 1 − ε.
In other words, define val(A) = supw∈A∗ PA(w), is val(A) = 1?
-
2The value 1 problem
This talk is about the value 1 problem:
INPUT: A a probabilistic automatonOUTPUT: for all ε > 0, there exists w ∈ A∗, PA(w) ≥ 1 − ε.
In other words, define val(A) = supw∈A∗ PA(w), is val(A) = 1?
It is undecidable (Gimbert and Oualhadj, 2010).
But to what extent?
-
3Terminology
0
positive: > 0
12
threshold: ≥ 12
1
almost-sure: = 1
limit-sure: lim = 1
-
4A Research Program
Construct an algorithm to decide the value 1 problem,
which is often correct.
-
4A Research Program
Construct an algorithm to decide the value 1 problem,
which is often correct.
Quantify how often.
-
4A Research Program
Construct an algorithm to decide the value 1 problem,
which is often correct.
Quantify how often.
Argue that you cannot do more often than that.
-
4Outline
1 Theory
A first attempt: get rid of numerical values
A second attempt: the Markov Monoid Algorithm
On the optimality of the Markov Monoid Algorithm
2 Practice: ACMÉ
-
4Outline
1 Theory
A first attempt: get rid of numerical values
A second attempt: the Markov Monoid Algorithm
On the optimality of the Markov Monoid Algorithm
2 Practice: ACMÉ
-
4Outline
1 Theory
A first attempt: get rid of numerical values
A second attempt: the Markov Monoid Algorithm
On the optimality of the Markov Monoid Algorithm
2 Practice: ACMÉ
-
5At first thought
Does the undecidability come fromthe numerical values?
-
5At first thought
Does the undecidability come fromthe numerical values?
Consider numberless probabilistic automata:
1 2
a
a
a
Two decision problems:
for all ∆, val(A[∆]) = 1,
there exists ∆, such that val(A[∆]) = 1.
-
6No future (in this direction)
Theorem (F., Horn, Gimbert and Oualhadj)
There is no algorithm such that:
On input A (a numberless probabilistic automaton),
if for all ∆, val(A[∆]) = 1 then “YES”,
if for all ∆, val(A[∆]) < 1 then “NO”,
anything in the other cases.
-
6Outline
1 Theory
A first attempt: get rid of numerical values
A second attempt: the Markov Monoid Algorithm
On the optimality of the Markov Monoid Algorithm
2 Practice: ACMÉ
-
7An example
0 1 F⊥
a a
a, ba, bb
a
b
b
-
7An example
0 1 F⊥
a a
a, ba, bb
a
b
b
〈a〉 0 1 F⊥
-
7An example
0 1 F⊥
a a
a, ba, bb
a
b
b
〈a〉 0 1 F⊥
〈b〉 0 1 F⊥
-
7An example
0 1 F⊥
a a
a, ba, bb
a
b
b
〈a〉 0 1 F⊥
〈a♯〉 0 1 F⊥
-
7An example
0 1 F⊥
a a
a, ba, bb
a
b
b
〈b〉 0 1 F⊥
〈a♯〉 0 1 F⊥
〈a♯ · b〉 0 1 F⊥
-
7An example
0 1 F⊥
a a
a, ba, bb
a
b
b
〈a♯ · b〉 0 1 F⊥
〈(a♯ · b)♯〉 0 1 F⊥
-
7An example
0 1 F⊥
a a
a, ba, bb
a
b
b
〈a〉 0 1 F⊥
〈b〉 0 1 F⊥
〈a♯〉 0 1 F⊥
〈a♯ · b〉 0 1 F⊥
〈(a♯ · b)♯〉 0 1 F⊥
-
8Stabilization monoids (Colcombet)
This is an algebraic structure with two operations:
binary composition
stabilization, denoted ♯
With some natural axioms:
(u♯)♯ = u♯
u · (v · u)♯ = (u · v)♯ · u
-
9Boolean matrices representations
0 1 F⊥
a a
a, ba, bb
a
b
b
〈a〉 =
1 0 0 0
0 1 1 0
0 0 1 0
0 0 0 1
〈b〉 =
1 0 0 0
1 0 0 0
0 1 1 0
0 0 0 1
I · 〈u〉 · F = 1 if and only if PA(u) > 0
-
10Defining stabilization
1 2
a
a
a
〈a〉 =
(1 1
0 1
)
In 〈a〉, the state 1 is transient and the state 2 is recurrent.
-
10Defining stabilization
1 2
a
a
a
〈a〉 =
(1 1
0 1
)〈a♯〉 =
(0 1
0 1
)
In 〈a〉, the state 1 is transient and the state 2 is recurrent.
-
10Defining stabilization
1 2
a
a
a
〈a〉 =
(1 1
0 1
)〈a♯〉 =
(0 1
0 1
)
In 〈a〉, the state 1 is transient and the state 2 is recurrent.
M♯(s, t) =
{1 if M(s, t) = 1 and t recurrent in M,0 otherwise.
-
11The Markov Monoid Algorithm
Compute a monoid inside the finite monoid MQ×Q({0, 1},∨,∧).
Compute 〈a〉 for a ∈ A:
〈a〉(s, t) =
{1 if PA(s
a−→ t) > 0,
0 otherwise.
Close under product and stabilization.
-
11The Markov Monoid Algorithm
Compute a monoid inside the finite monoid MQ×Q({0, 1},∨,∧).
Compute 〈a〉 for a ∈ A:
〈a〉(s, t) =
{1 if PA(s
a−→ t) > 0,
0 otherwise.
Close under product and stabilization.
If there exists a matrix M such that
∀t ∈ Q, M(s0, t) = 1 ⇒ t ∈ F
then “A has value 1”, otherwise “A does not have value 1”.
-
12Correctness
Theorem
If there exists a matrix M such that
∀t ∈ Q, M(s0, t) = 1 ⇒ t ∈ F
then A has value 1.
-
12Correctness
Theorem
If there exists a matrix M such that
∀t ∈ Q, M(s0, t) = 1 ⇒ t ∈ F
then A has value 1.
But the value 1 problem is undecidable, so. . .
-
13No completeness
0
L1
L2
R1
R2a
b, 12
a, 1 − xb
a, x
a, b
b, 12
a, x b
a, 1 − x
a, b
Left and right parts are symmetric, so for all M:
M(0,L2) = 1 ⇐⇒ M(0,R2) = 1.
Yet: it has value 1 if and only if x > 12.
-
14A leak
1
2
3
b
a
b
a
a, b
a
-
14A leak
1
2
3
b
a
b
a
a, b
a 〈a♯ · b〉
1
2
3
-
14A leak
1
2
3
b
a
b
a
a, b
a 〈a♯ · b〉
1
2
3
ε
There is a leak from 1 to 2.
-
14A leak
1
2
3
b
a
b
a
a, b
a 〈a♯ · b〉
1
2
3
There is a leak from 1 to 2.
Definition
An automaton A is leaktight if it has no leak.
-
15Leaktight automata
Theorem (F.,Gimbert and Oualhadj 2012)
The algorithm is complete for leaktight automata.
Hence, the value 1 problem is decidable for leaktight automata.
The proof relies on Simon’s factorization forest theorem.
-
16Other decidable subclasses: in 2012
leaktight
[FGO12]simple
[CT12]
structurally
simple
[CT12]
♯-acyclic
[GO10]deterministic
-
17Other decidable subclasses: today
leaktight
[FGO12]simple
[CT12]
structurally
simple
[CT12]
♯-acyclic
[GO10]deterministic
-
18Corollary
So far,
the Markov Monoid Algorithm is
the most correct algorithm known
to solve the value 1 problem.
-
18Corollary
So far,
the Markov Monoid Algorithm is
the most correct algorithm known
to solve the value 1 problem.
But for how long?
-
18Outline
1 Theory
A first attempt: get rid of numerical values
A second attempt: the Markov Monoid Algorithm
On the optimality of the Markov Monoid Algorithm
2 Practice: ACMÉ
-
19What it misses: different convergence speeds
0
L1
L2
R1
R2a
b, 12
a, 14b
a, 34
a, b
b, 12
a, 34 b
a, 14
a, b
limn
PA
(b(anb)f (n)
)= 1 if and only if lim
nf (n) ·
(3
4
)n= ∞ ,
-
19What it misses: different convergence speeds
0
L1
L2
R1
R2a
b, 12
a, 14b
a, 34
a, b
b, 12
a, 34 b
a, 14
a, b
limn
PA
(b(anb)f (n)
)= 1 if and only if lim
nf (n) ·
(3
4
)n= ∞ ,
so f (n) = 2n works but f (n) = n does not.
-
20A characterization
Ã∗ is the space of prostochastic words.
A∗ = Ã∗[0] ( Ã∗[1] ( Ã∗[2] ( · · · ( Ã∗ .
Lemma
The following are equivalent:
The value 1 problem over finite words,
The emptiness problem over prostochastic words.
-
20A characterization
Ã∗ is the space of prostochastic words.
A∗ = Ã∗[0] ( Ã∗[1] ( Ã∗[2] ( · · · ( Ã∗ .
Lemma
The following are equivalent:
The value 1 problem over finite words,
The emptiness problem over prostochastic words.
Theorem
1 The Markov Monoid Algorithm answers “YES” if and only if
there exists x ∈ Ã∗[1] accepted by A,
2 The following problem is undecidable: determine whether there
exists x ∈ Ã∗[2] accepted by A.
-
21Prostochastic words
Definition
(un)n∈N converges if for every A, the limit limn PA(un) exists.
-
21Prostochastic words
Definition
(un)n∈N converges if for every A, the limit limn PA(un) exists.
Definition
Two (converging) sequences (un)n∈N and (vn)n∈N are equivalent if forevery A,
limn
PA(un) > 0 ⇐⇒ limn
PA(vn) > 0 .
-
21Prostochastic words
Definition
(un)n∈N converges if for every A, the limit limn PA(un) exists.
Definition
Two (converging) sequences (un)n∈N and (vn)n∈N are equivalent if forevery A,
limn
PA(un) > 0 ⇐⇒ limn
PA(vn) > 0 .
Definition
A prostochastic word is an equivalence class of converging sequences.
-
22The ω operators
Definition
Let u be a converging sequence.
uω1 is the converging sequence (un!n )n∈N.
-
22The ω operators
Definition
Let u be a converging sequence.
uω1 is the converging sequence (un!n )n∈N.
Definition
Let u be a converging sequence.
uωk is the converging sequence (u(n!)kn )n∈N.
-
22The ω operators
Definition
Let u be a converging sequence.
uω1 is the converging sequence (un!n )n∈N.
Definition
Let u be a converging sequence.
uωk is the converging sequence (u(n!)kn )n∈N.
Example
The prostochastic words (aω1 b)ω1 and (aω1 b)ω2 are not equal.
-
23An equivalent characterization
Theorem
The Markov Monoid Algorithm answers “YES” if and only if there
exists a regular sequence (un)n∈N of finite words such thatlimn PA(un) = 1.
The regular sequences are described by the following grammar:
u = a | u · u | (unn)n∈N .
-
24Corollary
In some sense,
the Markov Monoid Algorithm is
the most correct algorithmto solve the value 1 problem.
-
24Outline
1 Theory
A first attempt: get rid of numerical values
A second attempt: the Markov Monoid Algorithm
On the optimality of the Markov Monoid Algorithm
2 Practice: ACMÉ
-
25ACMÉ
The tool ACME (Automata with Counters, Monoids and
Equivalence) has been written in OCaml by Nathanaël
Fijalkow and Denis Kuperberg.
**********************************
Statistics on the Markov Monoid Algorithm:
**********************************
Over 1000 Automata of size 7 with 15 transitions:
540 are leaktight and do not have value 1
133 are leaktight and have value 1
17 are not leaktight and may have value 1
310 are not leaktight and have value 1
TheoryA first attempt: get rid of numerical valuesA second attempt: the Markov Monoid AlgorithmOn the optimality of the Markov Monoid Algorithm
Practice: ACMÉ
