Turing Machines...
Transcript of Turing Machines...
Turing MachinesDecidability
Master Informatique
Université Paris 5 – René Descartes
2016
Master Info. Complexity – Turing 1/24
Some General KnowledgeAlan Mathison Turing
• UK, 1912 – 1954• Mathematician, computer scientist,
cryptanalyst• Most famous works:
I Computation model (« TuringMachines »)
I Work on Enigma during WWIII Imitation Game (« Turing Test »)
• Father of theoretical CS and AI• Turing Award ' Nobel Award for CS
Master Info. Complexity – Turing 2/24
Outline
Turing MachinesIntroduction to Turing MachinesTuring Machines and Decision ProblemsNon-DeterminismChurch-Turing Thesis
DecidabilityRecognizable and Decidable ProblemsHalting Problem and Reductions
Master Info. Complexity – Turing 3/24
Turing Machines
Computing Machine [Turing 1936]
• Abstract model to compute any « calculable » decimalnumberi.e. any number which can be computed with a finiteamount of resources• Proposed years before the apparition of computer, but
quite a faithful abstraction of modern machines• Roughly speaking,
I tape (sequence of squares) browsed by a reader/writer:memory of a computer
I transition function: processor of a computer
Master Info. Complexity – Turing 4/24
Definition of Turing Machines
A Turing machine is a tuple 〈Q,Γ, B,Σ, q0, δ, F 〉 with:• Q = {q0, q1, . . . , qm}, a finite set of states• Γ, the finite set of symbols used by the machine
(vocabulary)• B ∈ Γ, a particular symbol called blank• Σ ⊆ Γ, the input vocabulary• q0 ∈ Q, the initial state of the machine• δ : Q× Σ→ Q× Γ× {L,R}, the transition function• F ⊆ Q, the set of final states
Master Info. Complexity – Turing 5/24
Transition Function
δ : Q× Σ→ Q× Γ× {L,R}
• Input: a pair (current state, symbol on the tape) calledconfiguration• Output: a tuple (next state, symbol to write, move)
Master Info. Complexity – Turing 6/24
Example of Turing Machines (1/2)Multiplying Integers by 2
Let us considerM = 〈Q,Γ, B,Σ, q0, δ, F 〉 with:
• Q = {in_progress, done}• Γ = {0, 1, B}• Σ = {0, 1}
• q0 = in_progress• δ as described in the table• F = done
Current state Current symbol Next State Write Movein_progress 0 in_progress 0 R
in_progress 1 in_progress 1 R
in_progress B done 0 R
done STOP
Master Info. Complexity – Turing 7/24
Example of Turing Machines (2/2)Multiplying Integers by 2
1 0 1 0 1 0 B B . . .
i_p
• Read 1 in state in_progress: write 1, move right, statein_progress
Master Info. Complexity – Turing 8/24
Example of Turing Machines (2/2)Multiplying Integers by 2
1 0 1 0 1 0 B B . . .
i_p
• Read 0 in state in_progress: write 0, move right, statein_progress
Master Info. Complexity – Turing 8/24
Example of Turing Machines (2/2)Multiplying Integers by 2
1 0 1 0 1 0 B B . . .
i_p
• Read 1 in state in_progress: write 1, move right, statein_progress
Master Info. Complexity – Turing 8/24
Example of Turing Machines (2/2)Multiplying Integers by 2
1 0 1 0 1 0 B B . . .
i_p
• Read 0 in state in_progress: write 0, move right, statein_progress
Master Info. Complexity – Turing 8/24
Example of Turing Machines (2/2)Multiplying Integers by 2
1 0 1 0 1 0 B B . . .
i_p
• Read 1 in state in_progress: write 1, move right, statein_progress
Master Info. Complexity – Turing 8/24
Example of Turing Machines (2/2)Multiplying Integers by 2
1 0 1 0 1 0 B B . . .
i_p
• Read 0 in state in_progress: write 0, move right, statein_progress
Master Info. Complexity – Turing 8/24
Example of Turing Machines (2/2)Multiplying Integers by 2
1 0 1 0 1 0 B B . . .
i_p
• Read B in state in_progress: write 0, move right, statedone
Master Info. Complexity – Turing 8/24
Example of Turing Machines (2/2)Multiplying Integers by 2
1 0 1 0 1 0 0 B . . .
d
• State done: stop
Master Info. Complexity – Turing 8/24
Accept vs Reject vs Loop
We consider a special class of Turing machines, with threepossible « results ». For an input x, we say that the TuringmachineM• accepts x ifM reaches the final state YES after a finite
number of steps (i.e. transitions)• rejects x ifM reaches the final state NO after a finite
number of steps• loops on x ifM never reaches a final state
Master Info. Complexity – Turing 9/24
Language of a Turing MachineWe call language of a Turing MachineM the set of all inputswhich are accepted byM
L(M) = {x ∈ Σ∗ | M stops on x andM(x) = YES}
ExampleIfM returns YES on inputs P ∈ N[X] with exactly one root,then L(M) = {P ∈ N[X] | P is a polynomial with exactly oneroot }
Language ' ProblemWe associate the language L(M) of a Turing MachineM to adecision problem P
x ∈ L(M) iff x is a positive instance of P
Master Info. Complexity – Turing 10/24
Language of a Turing MachineWe call language of a Turing MachineM the set of all inputswhich are accepted byM
L(M) = {x ∈ Σ∗ | M stops on x andM(x) = YES}
ExampleIfM returns YES on inputs P ∈ N[X] with exactly one root,then L(M) = {P ∈ N[X] | P is a polynomial with exactly oneroot }
Language ' ProblemWe associate the language L(M) of a Turing MachineM to adecision problem P
x ∈ L(M) iff x is a positive instance of P
Master Info. Complexity – Turing 10/24
(Non-)Deterministic Turing MachinesDefinitionGivenM a Turing machine and δ its transition function,M isdeterministic (DTM) if d is a mapping from any configuration(q, x) to a single image (q′, x′,m)Otherwise,M is non-deterministic (NDTM)
Solving a Decision Problem
• A DTM solves a decision problem P if the sequence oftransitions leads to the answer for any instance• Not so easy for NDTM: when the transition function has
several images, each of them must be checkedThe machine solves the problem if
I for each positive instance, at least one sequence oftransitions leads to a final state with the answer YES
I for each negative instance, every possible sequence oftransitions leads to a final state with the answer NO
Master Info. Complexity – Turing 11/24
(Non-)Deterministic Turing MachinesDefinitionGivenM a Turing machine and δ its transition function,M isdeterministic (DTM) if d is a mapping from any configuration(q, x) to a single image (q′, x′,m)Otherwise,M is non-deterministic (NDTM)
Solving a Decision Problem
• A DTM solves a decision problem P if the sequence oftransitions leads to the answer for any instance• Not so easy for NDTM: when the transition function has
several images, each of them must be checkedThe machine solves the problem if
I for each positive instance, at least one sequence oftransitions leads to a final state with the answer YES
I for each negative instance, every possible sequence oftransitions leads to a final state with the answer NO
Master Info. Complexity – Turing 11/24
Illustration: DTM vs NDTM
Deterministic(q0, x0)
(q1, x1)
. . .
(qn, xn)
δ
δ
δ
Non-Deterministic(q0, x0)
(q1, x1) . . . (qk, xk)
. . .. . . . . .
(qm, xm) (qn, xn)(qo, xo)
(qp, xp)
δ δ δ
δ
δ
δ
δ δ
δ
δ
Master Info. Complexity – Turing 12/24
Other Computing ModelsOther models have been proposed• Turing machines with several tapes/several
dimensions/several readers-writers• λ-calculus [Church 1936]• Kolmogorov-Uspensky machines
[Kolmogorov and Uspensky 1958]• Schönhage machines [Schönhage 1980]• . . .
Power of Turing machines: the Church-Turing ThesisEverything which can be computed with any of these modelscan be computed with a Turing machine
Turing completenessA system (computing model, programming language,. . . ) iscalled Turing complete if it can compute every possiblecomputable function
Master Info. Complexity – Turing 13/24
Other Computing ModelsOther models have been proposed• Turing machines with several tapes/several
dimensions/several readers-writers• λ-calculus [Church 1936]• Kolmogorov-Uspensky machines
[Kolmogorov and Uspensky 1958]• Schönhage machines [Schönhage 1980]• . . .
Power of Turing machines: the Church-Turing ThesisEverything which can be computed with any of these modelscan be computed with a Turing machine
Turing completenessA system (computing model, programming language,. . . ) iscalled Turing complete if it can compute every possiblecomputable function
Master Info. Complexity – Turing 13/24
Outline
Turing MachinesIntroduction to Turing MachinesTuring Machines and Decision ProblemsNon-DeterminismChurch-Turing Thesis
DecidabilityRecognizable and Decidable ProblemsHalting Problem and Reductions
Master Info. Complexity – Turing 14/24
Recognizability
Recognizability of a ProblemA problem P is called recognizable if there exists a TuringmachineM such that, for each instance i of P,M(i) answersYES iff i is a positive instance of P
Recognizability of a LanguageA language L is called recognizable if there exists a TuringmachineM such that L = L(M)
Prime Numbers
• Is n ∈ N a prime number?
This problem/language is recognizable: we can write analgorithm which answers YES when n is prime, and doesn’t careof non-prime numbers
Master Info. Complexity – Turing 15/24
Recognizability
Recognizability of a ProblemA problem P is called recognizable if there exists a TuringmachineM such that, for each instance i of P,M(i) answersYES iff i is a positive instance of P
Recognizability of a LanguageA language L is called recognizable if there exists a TuringmachineM such that L = L(M)
Prime Numbers
• Is n ∈ N a prime number?
This problem/language is recognizable: we can write analgorithm which answers YES when n is prime, and doesn’t careof non-prime numbers
Master Info. Complexity – Turing 15/24
DecidabilityDecidability of a ProblemA problem P is called decidable if there exists a Turing machineM such that, for each instance i of P,• M(i) answers YES when i is a positive instance of P• M(i) answers NO when i is a negative instance of P
Decidability of a ProblemA language L is called decidable if there exists a Turing machineM which accepts each x ∈ L and rejects each x /∈ LEquivalently: L is decidable iff both L and L̄ are recognizable
Prime Numbers
• Is n ∈ N a prime number?
This problem/language is decidable: we can write an algorithmwhich answers YES when n is prime, and NO otherwise
Master Info. Complexity – Turing 16/24
DecidabilityDecidability of a ProblemA problem P is called decidable if there exists a Turing machineM such that, for each instance i of P,• M(i) answers YES when i is a positive instance of P• M(i) answers NO when i is a negative instance of P
Decidability of a ProblemA language L is called decidable if there exists a Turing machineM which accepts each x ∈ L and rejects each x /∈ LEquivalently: L is decidable iff both L and L̄ are recognizable
Prime Numbers
• Is n ∈ N a prime number?
This problem/language is decidable: we can write an algorithmwhich answers YES when n is prime, and NO otherwise
Master Info. Complexity – Turing 16/24
Complement of a Decision Problem
DefinitionA problem P1 is called the complement of P2 L(P1) = L(P2).Equivalently:• P1 and P2 are defined on the same set of instances• Positive instances of P1 are negative instances of P2• Negative instances of P1 are positive instances of P2
Notation: P1 = P2
Prime Numbers
• Given n ∈ N, is there m ∈ N,m 6= 1,m 6= n, s.t. m dividesn?
This problem is the complement of the prime number problem.
Master Info. Complexity – Turing 17/24
Complement of a Decision Problem
DefinitionA problem P1 is called the complement of P2 L(P1) = L(P2).Equivalently:• P1 and P2 are defined on the same set of instances• Positive instances of P1 are negative instances of P2• Negative instances of P1 are positive instances of P2
Notation: P1 = P2
Prime Numbers
• Given n ∈ N, is there m ∈ N,m 6= 1,m 6= n, s.t. m dividesn?
This problem is the complement of the prime number problem.
Master Info. Complexity – Turing 17/24
Halting Problem
Is there a Turing machine H which has two parameters:• A Turing machineM• An input i ofM
and which returns• YES whenM stops on i• NO whenM loops on i
If there is such a machine, then Halting problem is decidable;otherwise, it is undecidable
Result from [Turing 1936]Halting is undecidable
Master Info. Complexity – Turing 18/24
Halting Problem
Is there a Turing machine H which has two parameters:• A Turing machineM• An input i ofM
and which returns• YES whenM stops on i• NO whenM loops on i
If there is such a machine, then Halting problem is decidable;otherwise, it is undecidable
Result from [Turing 1936]Halting is undecidable
Master Info. Complexity – Turing 18/24
Reducing a Problem to Another
Functional ReductionA functional reduction f is a total computable function from aproblem P1 to a problem P2 such that, for any instance i of P1,i is a positive instance iff f(i) is a positive instance of P2Notation:
P1 ≤f P2
TheoremIf P1 ≤f P2 and P1 is undecidable, then P2 is undecidable
Master Info. Complexity – Turing 19/24
Reducing a Problem to Another
Functional ReductionA functional reduction f is a total computable function from aproblem P1 to a problem P2 such that, for any instance i of P1,i is a positive instance iff f(i) is a positive instance of P2Notation:
P1 ≤f P2
TheoremIf P1 ≤f P2 and P1 is undecidable, then P2 is undecidable
Master Info. Complexity – Turing 19/24
(Trivial) Example of FunctionalReduction
• P1: Given n ∈ N, is n even?• P2: Given n ∈ N, is n odd?• f : N→ N is defined by f(n) = n+ 1
For a positive instance i of P1, f(i) is a positive instance of P2.For a negative instance i of P1, f(i) is a negative instance of P2.So P1 ≤f P2Here we also have P2 ≤f P1; this is trivial since these aredecidable problems.
Master Info. Complexity – Turing 20/24
(Trivial) Example of FunctionalReduction
• P1: Given n ∈ N, is n even?• P2: Given n ∈ N, is n odd?• f : N→ N is defined by f(n) = n+ 1
For a positive instance i of P1, f(i) is a positive instance of P2.For a negative instance i of P1, f(i) is a negative instance of P2.
So P1 ≤f P2Here we also have P2 ≤f P1; this is trivial since these aredecidable problems.
Master Info. Complexity – Turing 20/24
(Trivial) Example of FunctionalReduction
• P1: Given n ∈ N, is n even?• P2: Given n ∈ N, is n odd?• f : N→ N is defined by f(n) = n+ 1
For a positive instance i of P1, f(i) is a positive instance of P2.For a negative instance i of P1, f(i) is a negative instance of P2.So P1 ≤f P2
Here we also have P2 ≤f P1; this is trivial since these aredecidable problems.
Master Info. Complexity – Turing 20/24
(Trivial) Example of FunctionalReduction
• P1: Given n ∈ N, is n even?• P2: Given n ∈ N, is n odd?• f : N→ N is defined by f(n) = n+ 1
For a positive instance i of P1, f(i) is a positive instance of P2.For a negative instance i of P1, f(i) is a negative instance of P2.So P1 ≤f P2Here we also have P2 ≤f P1; this is trivial since these aredecidable problems.
Master Info. Complexity – Turing 20/24
A Less Trivial Example
Language
L1 = {M | M halts on ε}
with ε the empty word. Is L1 decidable?
Proof of UndecidabilityWe need a mapping f : (M, x) 7→ M′ s.t. (M, x) ∈ Halting iffM′ ∈ L1• M′x(y) =M(x.y) with x.y the concatenation of words• M′x(ε) =M(x), so obviously:
I IfM halts on x, thenM′x halts on ε
I IfM loops on x, thenM′x loops on ε
Master Info. Complexity – Turing 21/24
A Less Trivial Example
Language
L1 = {M | M halts on ε}
with ε the empty word. Is L1 decidable?
Proof of UndecidabilityWe need a mapping f : (M, x) 7→ M′ s.t. (M, x) ∈ Halting iffM′ ∈ L1
• M′x(y) =M(x.y) with x.y the concatenation of words• M′x(ε) =M(x), so obviously:
I IfM halts on x, thenM′x halts on ε
I IfM loops on x, thenM′x loops on ε
Master Info. Complexity – Turing 21/24
A Less Trivial Example
Language
L1 = {M | M halts on ε}
with ε the empty word. Is L1 decidable?
Proof of UndecidabilityWe need a mapping f : (M, x) 7→ M′ s.t. (M, x) ∈ Halting iffM′ ∈ L1• M′x(y) =M(x.y) with x.y the concatenation of words• M′x(ε) =M(x), so obviously:
I IfM halts on x, thenM′x halts on ε
I IfM loops on x, thenM′x loops on ε
Master Info. Complexity – Turing 21/24
Other Method to Prove Undecidability
« Algorithmic » ProofIf P1 is known to be undecidable, we can prove that P2 isundecidable with an algorithm which computes P1 using only« simple » steps and calls to P2
Remember: L1 ={M | M halts on ε}.Suppose thatML1 isa Turing Machinewhich decides L1.
Algorithm 1 HaltingInput: M, x = x0x1 . . . xn
if x = ε thenreturn ML1(M)
elsex′ = x1 . . . xnM′ =M with q0 replaced by δ(q0, x0)return Halting(M′, x′)
end if
Master Info. Complexity – Turing 22/24
Other Method to Prove Undecidability
« Algorithmic » ProofIf P1 is known to be undecidable, we can prove that P2 isundecidable with an algorithm which computes P1 using only« simple » steps and calls to P2
Remember: L1 ={M | M halts on ε}.Suppose thatML1 isa Turing Machinewhich decides L1.
Algorithm 2 HaltingInput: M, x = x0x1 . . . xn
if x = ε thenreturn ML1(M)
elsex′ = x1 . . . xnM′ =M with q0 replaced by δ(q0, x0)return Halting(M′, x′)
end if
Master Info. Complexity – Turing 22/24
Other Method to Prove Undecidability
« Algorithmic » ProofIf P1 is known to be undecidable, we can prove that P2 isundecidable with an algorithm which computes P1 using only« simple » steps and calls to P2
Remember: L1 ={M | M halts on ε}.Suppose thatML1 isa Turing Machinewhich decides L1.
Algorithm 3 HaltingInput: M, x = x0x1 . . . xn
if x = ε thenreturn ML1(M)
elsex′ = x1 . . . xnM′ =M with q0 replaced by δ(q0, x0)return Halting(M′, x′)
end if
Master Info. Complexity – Turing 22/24
Other Method to Prove Undecidability
« Algorithmic » ProofIf P1 is known to be undecidable, we can prove that P2 isundecidable with an algorithm which computes P1 using only« simple » steps and calls to P2
Remember: L1 ={M | M halts on ε}.Suppose thatML1 isa Turing Machinewhich decides L1.
Algorithm 4 HaltingInput: M, x = x0x1 . . . xn
if x = ε thenreturn ML1(M)
elsex′ = x1 . . . xnM′ =M with q0 replaced by δ(q0, x0)return Halting(M′, x′)
end if
Master Info. Complexity – Turing 22/24
How to Prove Decidability?• Prove P1 ≤f P2 with P2 decidable, then P1 is decidable
• Write an algorithm which solves your problem with only« simple » steps!
Of course, the more complex is the algorithm, the less easy it isto be sure that it is correct :(
Example: Is n ∈ N a prime number?
Algorithm 5 PrimeInput: n ∈ N
for x ∈ {2, . . . , b√nc} do
if ∃y ∈ N s.t. n = x× y thenreturn false
end ifend forreturn true
Master Info. Complexity – Turing 23/24
How to Prove Decidability?• Prove P1 ≤f P2 with P2 decidable, then P1 is decidable• Write an algorithm which solves your problem with only
« simple » steps!Of course, the more complex is the algorithm, the less easy it isto be sure that it is correct :(
Example: Is n ∈ N a prime number?
Algorithm 6 PrimeInput: n ∈ N
for x ∈ {2, . . . , b√nc} do
if ∃y ∈ N s.t. n = x× y thenreturn false
end ifend forreturn true
Master Info. Complexity – Turing 23/24
How to Prove Decidability?• Prove P1 ≤f P2 with P2 decidable, then P1 is decidable• Write an algorithm which solves your problem with only
« simple » steps!Of course, the more complex is the algorithm, the less easy it isto be sure that it is correct :(
Example: Is n ∈ N a prime number?
Algorithm 7 PrimeInput: n ∈ N
for x ∈ {2, . . . , b√nc} do
if ∃y ∈ N s.t. n = x× y thenreturn false
end ifend forreturn true
Master Info. Complexity – Turing 23/24
References
A. M. Turing, On computable numbers, with an applicationto the Entscheidungsproblem. Proceedings of the LondonMathematical Society, 1936.
A. Church, An unsolvable problem of elementary numbertheory. American Journal of Mathematics, 1936.
A. N. Kolmogorov and V. Uspensky, On the definition of analgorithm. Uspekhi Mat. Naut, 1958.
A. Schönhage, Storage modification machines. SIAMJournal on Computing, 1980.
Master Info. Complexity – Turing 24/24