CSCI 3130: Formal languages and automata theory

Andrej Bogdanov

http://www.cse.cuhk.edu.hk/~andrejb/csc3130

The Chinese University of Hong Kong

Decidable andundecidable languages

Fall 2011

Problems about automata

• We can formulate this question as a language:

q0 q1

b

b

aa

Does accept input abb?

ADFA = { 〈 D, w 〉 : D is a DFA that accepts input w}

((q0,q1)(a,b)((q0,a,q0)(q0,b,q1)(q1,a,q0)(q1,b,q1)(q0)(q1))(abb)

D = (Q, , , q0, F) w

Is ADFA decidable?

Problems about automata

ADFA = { 〈 D, w 〉 : D is a DFA that accepts input w}

pseudocode:

On input 〈 D, w 〉 , where D = (Q, , , q0, F):

Set q := q0

For i := 1 to length(w):

q := (q, wi)

If q∈F accept, else reject

TM description:

On input 〈 D, w 〉 , where D is a DFA, w is a string

Simulate D on input w

If simulation ends in acc state,accept. Otherwise, reject.

Problems about automata

ADFA = { 〈 D, w 〉 : D is a DFA that accepts input w}

Turing Machine details:

Check input is in correct format.(Transition function is complete, no duplicate transitions)

Perform simulation:

((q0,q1)(a,b)((q0,a,q0)(q0,b,q1)(q1,a,q0)(q1,b,q1)(q0)(q1))(abb). .state input symbol

. .. .

qacc

Problems about automata

ADFA = { 〈 D, w 〉 : D is a DFA that accepts input w}

Turing Machine details:

Check input is in correct format.(Transition function is complete, no duplicate transitions)

Perform simulation:

Put markers on start state of D and first symbol of w

Until marker for w reaches last symbol:

Update both markers

If state marker is on accepting state, accept. Else reject.

decidable

Acceptance problems about automata

ADFA = { 〈 D, w 〉 : D is a DFA that accepts input w}

ANFA = { 〈 N, w 〉 : N is an NFA that accepts w}

AREX = { 〈 R, w 〉 : R is a regular expression that generates w}

Which of these is decidable?

decidable

Acceptance problems about automata

• The following TM decides ADFA:

ADFA = { 〈 D, w 〉 : D is a DFA that accepts input w} decidable

M :=On input 〈 D, w 〉 , where D is a DFA and w is a string Simulate D on input w If the simulation ends in acc state of D, accept. If it doesn’t, reject.

Acceptance problems about automata

• The following TM decides ANFA:

ANFA = { 〈 N, w 〉 : N is an NFA that accepts input w} decidable

N :=On input 〈 N, w 〉 , where N is an NFA and w is a stringConvert N to a DFA D using the conversion procedure from Lecture 2Run the TM M for ADFA on input 〈 D, w 〉 If M accepts, accept. Otherwise, reject.

Acceptance problems about automata

• The following TM decides AREX:

AREX = { 〈 R, w 〉 : R is a regular expression that generates w} decidable

P :=On input 〈 R, w 〉 , where R is a reg exp and w is a stringConvert R to an NFA N using the conversion procedure from Lecture 4Run the TM N for ANFA on input 〈 N, w 〉 If N accepts, accept. Otherwise, reject.

Other problems about automata

MINDFA = { 〈 D 〉 : D is a minimal DFA}

On input 〈 D 〉 , where D is a DFA

decidable

R :=

Run the distinguishable states algorithm from Lecture 7 on DIf every pair of states is distinguishable, accept.Otherwise, reject.

• The following TM decides MINDFA:

Other problems about automata

EQDFA = { 〈 D1, D2 〉 : D1, D2 are DFAs and L(D1) = L(D2)}

On input 〈 D1, D2 〉 , where D1 and D2 are DFAs

decidable

S :=

Run the DFA minimization algorithm from Lecture 7 on D1 to obtain a DFA D1’

• The following TM decides EQDFA:

If D1’ = D2’ accept, otherwise reject.

Run the DFA minimization algorithm from Lecture 7 on D2 to obtain a DFA D2’

Other problems about automata

EDFA = { 〈 D 〉 : D is a DFAs and L(D) is empty}

On input 〈 D 〉 , where D is a DFA

decidable

T :=

Run the TM S for EQDFA on input 〈 D, 〉

• The following TM decides EDFA:

If S accepts accept, otherwise reject.

Problems about context-free grammars

ACFG = { 〈 G, w 〉 : G is a CFG that generates w}

On input 〈 G, w 〉 , where G is a CFG and w is a string

decidable

V :=

If the CYK algorithm produces a parse tree, accept.

Eliminate the nullable and unit productions from GConvert G to Chomsky Normal Form

Run the Cocke-Younger-Kasami algorithm on 〈 G, w 〉

Otherwise, reject.

Decidability of context-free languages

Every context-free language is decidable.

Let L be a context-free language.

There is a CFG G for L.

The following TM decides L:

MG :=

On input w,

Run TM V on input 〈 G, w 〉 .If V accepts accept, otherwise reject.

Are all languages about CFGs decidable?

EQCFG = { 〈 G1, G2 〉 : G1 and G2 are context-free grammars that generate the same strings}

undecidable

What’s the difference between EQDFA and EQCFG?

To decide EQDFA, we minimized both DFAs

But there is no method that, given a CFG or PDA,produces a unique equivalent minimal CFG or PDA

The universal Turing Machine

and undecidability

Turing Machines versus computers

computerdata

programoutput

A computer is a machine that manipulates data according to a list of instructions.

How does a Turing Machine take a program as part of its input?

The Universal Turing Machine

Uinput x for M

program 〈 M 〉 whatever

M does on x

The universal TM U takes as inputs a program Mand a string x and simulates M on x

The program M itself is specified as a TM!

Turing Machines as strings

• This Turing Machine can be described by the string

〈 M 〉 = (q,qa,qr)(0,1)(0,1, )☐((q,q, / R☐ ☐ )

(q,qa,0/0R) (q,qr,1/1R))

(q)(qa)(qr)

M

q

qa

qr

☐/☐R0/0R

1/1R A Turing Machine is (Q, , , , q0, qacc, qrej)

The universal Turing Machine

U (q,qa,qr)(0,1)(0,1, 001

input w for Mprogram 〈 M 〉

U := On input 〈 M, w 〉 ,

Simulate M on input wIf M enters accept state, accept.If M enters reject state, reject.

Acceptance of Turing Machines

ATM = { 〈 M, w 〉 : M is a TM that accepts w}

U := On input 〈 M, w 〉 , Simulate M on input w

M accepts w M rejects w M loops on w

U accepts 〈 M, w 〉U rejects 〈 M, w 〉U loops on 〈 M, w 〉

TM U recognizes but does not decide ATM

Recognizing versus deciding

The language recognized by a TM is the set ofall inputs that make it acceptA TM decides language L if it recognizes L andhalts (does not loop) on every input

qacc qrej

accept reject loop

halt

Undecidability

• Turing’s Theorem:

• Before we show this, let’s observe one thing:

The language ATM is undecidable.

A Turing Machine M can be given its owndescription 〈 M 〉 as an input!

Turing’s theorem

A TM that decides ATM is so potent that it will

destroy itself.

Proof of Turing’s Theorem

• Proof by contradiction:

Suppose ATM is decidable.

Then there is a TM H that decides ATM:

H〈 M, w 〉

accept if M accepts wreject if M rejects w

or M loops on w

〈 M, 〈 M 〉〉

accept if M accepts 〈 M 〉reject if M rejects 〈 M 〉

or M loops on 〈 M 〉What happens when w = 〈 M 〉 ?

Proof of undecidability

H〈 M, 〈 M 〉〉

accept if M accepts 〈 M 〉reject if M rejects 〈 M 〉

or M loops on 〈 M 〉Let H’ be a TM that does the opposite of H

Hacc

rej

acc

rej

H ’ To go from H to H’,just switch its acceptand reject states

Proof of undecidability

H〈 M, 〈 M 〉〉

accept if M accepts 〈 M 〉reject if M rejects 〈 M 〉

or M loops on 〈 M 〉

H ’〈 M, 〈 M 〉〉

reject if M accepts 〈 M 〉

if M rejects 〈 M 〉 or M loops on 〈 M 〉

accept

Proof of undecidability

H ’〈 M, 〈 M 〉〉

reject if M accepts 〈 M 〉

if M rejects 〈 M 〉 or M loops on 〈 M 〉

accept

Let D be the following TM:

copy〈 M 〉 〈 M,

〈 M 〉〉

H ’

Proof of undecidability

D〈M〉 reject if M accepts

〈 M 〉

if M rejects 〈 M 〉 or M loops on 〈 M 〉

accept

What happens when M = D?

If D accepts 〈 D 〉then D rejects 〈 D 〉

If D rejects 〈 D 〉then D accepts 〈 D 〉

so D does not exist!

H never loops, so

D never loops either

Proof of undecidability: conclusion

• Proof by contradiction– We assumed ATM was decidable

– Then we built Turing Machines H, H ’, D– But D does not exist!

• Conclusion

The language ATM is undecidable.

What happened?

M1

M2

M3

…

0 1 00 …

acc rej rej acc

rej acc loop rej

rej loop rej rej

…

…

Tu

rin

g M

ach

ines

all possible inputs w

We can write an infinite table for every pair (M, w)

all

poss

ible

What happened?

M1

M2

M3

…

〈 M1 〉〈 M2 〉 〈 M4 〉…

acc loop rej acc

rej acc rej acc

loop rej loop rej

…

…

Now let’s look only at those w that describe a TM M

〈 M3 〉

What happened?

M1

M2…

〈 M1 〉〈 M2 〉 〈 M4 〉…

acc loop rej acc

rej acc rej acc …

…

If ATM is decidable, then TM D is in the table

〈 M3 〉

D rej rej acc rej

… …

What happened?

M1

M2…

〈 M1 〉〈 M2 〉 〈 M4 〉…

acc loop rej acc

rej acc rej acc …

…D does the opposite of the diagonal entries

〈 M3 〉

D rej rej acc rej

D〈M〉

reject if M accepts 〈 M 〉

accept if M rejects or loops on 〈 M 〉

What happened?

M1

M2

…〈 M1 〉〈 M2 〉 〈 M4 〉…

acc loop rej acc

rej acc rej acc …

…

We run into trouble when we look at (D, 〈 D 〉 )!

〈 M3 〉

D rej rej acc rej

〈 D 〉

loop

acc

?…

Unrecognizable languages

• How about languages that are not recognizable?

The language ATM is recognizable but not decidable.

ATM = { 〈 M, w 〉 : M is a TM that does not accept w}

= { 〈 M, w 〉 : M rejects or loops on input w}

The language ATM is not recognizable.

Unrecognizable languages

• Theorem

• We know ATM is recognizable, so if ATM were also,then ATM would be decidable

• But Turing’s Theorem says ATM is not decidable

If L and L are both recognizable, then L isdecidable.

Unrecognizable languages

• Proof idea

If L and L are both recognizable, then L isdecidable.

M rej/loop if w ∉ L

accept if w ∈ Lw

M ’ rej/loop if w ∈ L

accept if w ∉ Lw

w

accept

reject

Unrecognizable languages

Let M = TM for L, M ’ = TM for L

On input w,

Simulate M on input w. If it accepts, accept.

Simulate M ’ on input w. If it accepts, reject.

If L and L are both recognizable, then L isdecidable.

Problem: If M loops on w, we will never get to step 2!

①

②

Bounded simulation

• Turing Machine that decides L:

Let M = TM for L, M ’ = TM for L

On input w,

Do t transitions of M on w. If it accepts, accept.

Do t transitions of M’ on w. If it accepts, reject.

If L and L are both recognizable, then L isdecidable.

For t := 0 to infinity