8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
1/35
CSC 3130: Automata theory and formal languages
Undecidable problems for CFGsand descriptive complexity
MELJUN P. CORTES MBA MPA BSCS ACS
MELJUN CORTES
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
2/35
Decidable vs. undecidable
TMM accepts w
TMMaccepts some input
TMMand M acceptsame inputs
TMM accepts all inputs
undecidable
TMM halts on wPDAM accepts w
DFAM accepts wdecidable
PDA Paccepts all
inputsCFG Gis ambiguous
other kinds of problems?
?
DFAMaccepts allinputs
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
3/35
0q0l0o0t0u0s0
0o0q6o0t0u0s00o0k6q3t0u0s
0o0k6r0q0u0s00o0k6r0a0q1s
0o0k6r0qaa0
Computation is local
lotus
ootus
oktus
okrus
okras
okra
Mq6
q0
q3
q0
q1
qacc
The changes between rows occur in a 2x3 window
computation
tableau
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
4/35
Computation histories as strings
IfMhalts on w, We can represent thecomputation tableau by a string t over alphabetGQ{#, }0q0l0o0t0u0s0
0o0q6o0t0u0s00o0k6q3t0u0s
0o0k6r0q0u0s00o0k6r0a0q1s
0o0k6r0qaa0
q0lotus#oq6otus#...#okrqaa#
Maccepts w qaoccurs in string t
Mrejects w qadoes not occur in t
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
5/35
Undecidable problems for PDAs
Theorem
Proof: We will show that
ALLPDA= {P: Pis a PDA that accepts allinputs}
The languageALLPDAis undecidable.
IfALLPDAcan be decided, so canATM.
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
6/35
Undecidable problems for PDAs
M, w
reject
ifMaccepts w
accept
ifMrej/loops w
Areject if not
accept ifPaccepts all inputsP
AP
P accepts all inputs ifMrejects or loops on w
Pdoes not accept some input ifMaccepts w
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
7/35
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
8/35
Undecidability via computation histories
Task: Design a PDA Psuch that
Pcandidate computationhistory tofMon w
reject acceptinghistories
0q0l0o0t0u0s0
0o0q6o0t0u0s00o0k6q3t0u0s
0o0k6r0q0u0s0
0o0k6r0a0q1s0o0k6r0qaa0
Expect tof the form w1#w2#...#wk#
If w1q0w, accept t.
If tdoes not contain qa, accept t.
If two consecutive blocks wi#wi+1do not correspond to a propertransition ofM, accept t.
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
9/35
Implementing P
If w1q0w, accept t.
If two consecutive blocks wi#wi+1do notrepresent a valid transition ofM, accept t.
On input t:
Nondeterministically make one of the following choices
If tdoes not contain qa, accept t.
Look for the beginning of the ith block of t
Look in the first block w1of t
Look for the appearance of qa
0o0k6q3t0u0s #
0o0k6r0q0u0s0
wi#wi+1represents a valid transition if all 3x2windows correspond to possible transitions ofM
valid transition
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
10/35
Valid and invalid windows
6c3a0t0 0c6a0p00
6t3t0u0 0t6t0u00
valid window
invalid window
6t3t0u0 0t6t0q3 0
valid window
6t3q3u0 0t6a0q7 0
valid if d(q3, u) = (q7, a, R)
6q3t0u0 0k6t0q0 0
invalid window
6c3a0t0 0b6a0t00
valid window
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
11/35
Implementing P
To check this it is better to write tin
boustrophedon
wi#wi+1represent a valid transition ofM
0q0l0o0t0u0s00o0q6o0t0u0s00o0k6q3t0u0s
0o0k6r0q0u0s00o0k6r0a0q1s
0o0k6r0qaa0
q0lotus#oq6otus#...#okrqaa#
q0lotus#sutoq6o#...#okrqaa#
Alternate rows are written in reverse
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
12/35
Implementing P
0o0k6q3t0u0s #
0o0k6r0q0u0s0
proper transition
#okq3tus#suq0rko#
wi wi+1
Nondeterministically look for beginning of 3x2 window
wi#wi+1represent a valid transition ofM
#
Remember first row of window in state
Use stack to detect beginning of second row
Remember second row of window in state
If window is not valid, accept, otherwise reject.
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
13/35
The Post Correspondence Problem
Input: A set of tiles like this
Given an infinite supply of such tiles, can youmatch top and bottom?
bab
cc
c
ab
a
ab
baa
a
bab
cc
c
ab
a
ab
baa
a
c
ab
a
baba
a
baba
bab
e
bab
e
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
14/35
Undecidability of PCP
Theorem
Proof: We will show that
PCP= {D: Dis a collection of tiles thatcontains a top-bottom match}
The language PCPis undecidable.
If PCPcan be decided, so canATM.
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
15/35
Undecidability of PCP
Idea: Matches represent accepting histories
M, w T (collection of tiles)
IfMaccepts w, then Tcan be matched
IfMrej/loops on w, then Tcannot be matched
q0lotus#oq6otus#okq3t...#qa
q0
lotus#oq6
otus#okq3
r...#qa
e
q0lotus#q0l
oq6t
tu
us
s#
#
oq60
okq3o
o
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
16/35
Some technicalities
We will assume that Before accepting, TMMerases its tape
One of the PCP tiles is marked as a starting tile
These assumptions can be made without loss ofgenerality (we will see why later)
bab
cc
c
ab
a
ab
baa
a
a
baba
s
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
17/35
Undecidability of PCP
To decideATM, we construct these tiles for PCP
M, w T (collection of tiles)IfMaccepts w, then Tcan be matched
IfMrej/loops on w, then Tcannot be matched
e
q0w#
sa1qia3
b1b2b3for each valid
window of this form
a
afor all ainG{#, }
#qa
e
#
#
efinal tiles
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
18/35
q0
lotus#oq6
otus#...#oq1
#qa
q0lotus#oq6otus#...#oq1#qa
Undecidability of PCP
q0lotus#oq6otus#...#oq1
#qa
eq0w#
s a1qia3b1b2b3
aa
#qa
e
#
#
e
accepting computation history
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
19/35
Undecidability of PCP
IfMrejects on input w, then qr appears on bottomat some point, but it cannot be matched on top
IfMloops on w, then matching keeps going
forever
e
q0w#
sa1qia3
b1b2b3
a
a
#qa
e
#
#
e
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
20/35
A technicality
We assumed that one tile marked as starting tile
We can remove assumption by changing tiles abit
bab
cc
c
ab
a
baba
s
b*a*b*
*c*c
c*
*a*b
*a*
*b*a*b*a
*
starting tilebegins with *
ending tilematches last *
middle tiles
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
21/35
Ambiguity of CFGs
AMB= {G: Gis an ambiguousCFG}
Theorem
Proof: We will show that
The languageAMBis undecidable.
IfAMBcan be decided, so can PCP.
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
22/35
Ambiguity of CFGs
Proof:
Step 1: Number the tiles
T G
If Tcan be matched, then G isambiguous
If Tcannot be matched, then Gis unambiguous
(collection of tiles)
bab
cc
c
ab
a
ab
1 2 3
(CFG)
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
23/35
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
24/35
Ambiguity of CFGs
Each sequence of tiles gives two derivations
If the tiles match, these two derive the samestring
bab
cc
c
ab
1 2c
ab
2
S T babT1 babcT21 babcc221
S B ccB1 ccabB21 ccabab221
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
25/35
Ambiguity of CFGs
Argue by contradiction: If Gis ambiguous then ambiguity must look like this
T G
If Tcan be matched, then G isambiguous
If Tcannot be matched, then Gis unambiguous
(collection of tiles) (CFG)
S
T
Ta1 n1
ai ni
Ta2 n2
S
B
Bb1 m1
bj mj
Bb2 m2
Then n1...ni= m1mj
So there is a match
a1b1
a2b2
aibi
n1 n2 ni
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
26/35
Descriptive complexity
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
27/35
Roulette
In a game of roulette, you bet $1 on evenor odd
The outcome is a number between 1 and 36
If you guessed correctly, double your bet
Otherwise, you lose
17 6 5 16 5 2
11 8 31 18 7 4
5 2 29 8 1 12
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
28/35
Randomness
If we write Efor even, Ofor odd, what we saw is
It seems the wheel is crooked. If it wasnt wewould expect something more like
But both sequences have same probability! Whydoes one appear less random than the other?
OEOEOEOEOEOEOEOEOEOE
OOOEEOEOOEOEOOOEEEOE
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
29/35
Turing Machines with output
The goal of a Turing Machine with output is towrite something on the output tape and go intostate qhalt
M
output tape
0 1 0
work tape
0 1 0
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
30/35
Descriptive complexity
The descriptive complexity K(x)of xis theshortestdescriptionof any Turing Machine thatoutputs x
We will assume xis longAndrey Kolmogorov(1903-1987)
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
31/35
Example of descriptive complexity
Turing machine implementation:
x= OE...OE = (OE)n Repeat for nsteps:At odd step print OAt even step print E
Write nin binary on work tape
While work tape not equal to 0,Subtract 1from number on work tape
If number is odd, write OIf number is even, write E
(n= 1,000,000,000)
log2nstates
3states
15states
2states
log2n + 20K(x)
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
32/35
Bounds on descriptive complexity
Theorem 1
Proof: Let x= x1...xnand consider the followingTM:
For every xof length n, K(x)is at most O(n)
Write x1to output tape and move rightWrite x2to output tape and move right
Write xnto output tape and halt.
...
n+ O(1)
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
33/35
Descriptive complexity and randomness
Theorem 2For 99%of strings of length n, K(x) n10.
0 O(logn) n10
simple
strings111...1, OEOE...OE,
3.14159265, 1212321234321
random-looking
strings
n+O(1)
randomness-deficient
strings
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
34/35
Evaluating randomness
How do we know if the casino is crooked?
Idea: Compute K(sequence).
If much less than n, indicates sequence is not
random
17 6 5 16 5 2
11 8 31 18 11 13
5 2 29 8 1 12
12
14
12
8
31
4
8/10/2019 MELJUN CORTES AUTOMATA THEORY chapter19
35/35
Computing descriptive complexity
Proof:Suppose it is, fix nand consider this TM M:
Let x= output ofM, then
So (when nis large) we get K(x) > K(x),
i ibl !
It is not possible to compute K(x).
Output the first xof length n(in lexicographic order)such that K(x) n10
K(x) n10 K(x) |M| = log2n+ O(1)but