CD5560 FABER Formal Languages, Automata and Models of Computation Lecture 13

1

CD5560

FABER

Formal Languages, Automata and Models of Computation

Lecture 13

Mälardalen University

2010

2

Content

Alan Turing and Hilbert Program Universal Turing Machine Chomsky Hierarchy DecidabilityReducibilityUncomputable FunctionsRice’s TheoremInteractive Computing, Persistent TM’s (Dina Goldin)

3

http://www.turing.org.uk/turing/

Who was Alan Turing?

Founder of computer science,

mathematician,

philosopher,

codebreaker,

visionary man before his time.

http://www.cs.usfca.edu/www.AlanTuring.net/turing_archive/index.html- Jack

Copeland and Diane Proudfoot http://www.turing.org.uk/turing/ The Alan Turing Home Page

Andrew Hodges

http://www.turing.org.uk/turing/

4

Alan Turing

1912 (23 June): Birth, London

1926-31: Sherborne School

1930: Death of friend Christopher Morcom

1931-34: Undergraduate at King's College, Cambridge University

1932-35: Quantum mechanics, probability, logic

1935: Elected fellow of King's College, Cambridge

1936: The Turing machine, computability, universal machine

1936-38: Princeton University. Ph.D. Logic, algebra, number theory

1938-39: Return to Cambridge. Introduced to German Enigma cipher machine

1939-40: The Bombe, machine for Enigma decryption

1939-42: Breaking of U-boat Enigma, saving battle of the Atlantic

5

Alan Turing

1943-45: Chief Anglo-American crypto consultant. Electronic work.

1945: National Physical Laboratory, London

1946: Computer and software design leading the world.

1947-48: Programming, neural nets, and artificial intelligence

1948: Manchester University

1949: First serious mathematical use of a computer

1950: The Turing Test for machine intelligence

1951: Elected FRS. Non-linear theory of biological growth

1952: Arrested as a homosexual, loss of security clearance

1953-54: Unfinished work in biology and physics

1954 (7 June): Death (suicide) by cyanide poisoning, Wilmslow, Cheshire.

6

Hilbert’s Program, 1900

Hilbert’s hope was that mathematics would be reducible to finding proofs (manipulating the strings of symbols) from a fixed system of axioms, axioms that everyone could agree were true.

Can all of mathematics be made algorithmic, or will there always be new problems that outstrip any given algorithm, and so require creative acts of mind to solve?

7

TURING MACHINES

8

Turing’s "Machines". These machines are humans who calculate.

(Wittgenstein)

A man provided with paper, pencil, and rubber, and subject to strict discipline, is in effect a universal machine.

(Turing)

9

............Tape

Read-Write head

Control Unit

Standard Turing Machine

10

............

Read-Write head

No boundaries -- infinite length

The head moves Left or Right

The Tape

11

............

Read-Write head

1. Reads a symbol

2. Writes a symbol

3. Moves Left or Right

The head at each time step:

12

ExampleTime 0

............ a a cb

Time 1............ a b k c

1. Reads a2. Writes k3. Moves Left

13

Head starts at the leftmost position

of the input string

............

Blank symbol

head

a b ca

Input string

The Input String

#####

14

1q 2qLba ,

Read WriteMove Left

1q 2qRba ,

Move Right

States & Transitions

15

............ # a b caTime 1

1q 2qRba ,

............ a b cbTime 2

1q

2q

# # # #

# # # # #

16

Determinism

1q

2qRba ,

Allowed Not Allowed

3qLdb ,

1q

2qRba ,

3qLda ,

No lambda transitions allowed in standard TM!

Turing Machines are deterministic

17

Formal Definitions for

Turing Machines

18

Transition Function

1q 2qRba ,

),,(),( 21 Rbqaq

19

Turing Machine

),#,,,,,( 0 FqQM

Transition

functionInitial

stateblank

Final

states

States

Input

alphabetTape

alphabet

20

Universal Turing Machine

21

A limitation of Turing Machines:

Better are reprogrammable machines.

Turing Machines are “hardwired”

they execute

only one program

22

Solution: Universal Turing Machine

• Reprogrammable machine

• Simulates any other Turing Machine

Characteristics:

23

Universal Turing Machine

simulates any other Turing Machine M

Input of Universal Turing Machine

• Description of transitions of M

• Initial tape contents of M

24

Universal

Turing

Machine

Description of Three tapes

MTape Contents of

Tape 2

State of M

Tape 3

M

Tape 1

25

We describe Turing machine

as a string of symbols:

We encode as a string of symbols

M

M

Description of M

Tape 1

26

Alphabet Encoding

Symbols: a b c d

Encoding: 1 11 111 1111

27

State Encoding

States: 1q 2q 3q 4q

Encoding: 1 11 111 1111

Head Move Encoding

Move:

Encoding:

L R

1 11

28

Transition Encoding

Transition: ),,(),( 21 Lbqaq

Encoding: 10110110101

separator

29

Machine Encoding

Transitions:

),,(),( 21 Lbqaq

Encoding:

10110110101

),,(),( 32 Rcqbq

110111011110101100

separator

30

Tape 1 contents of Universal Turing Machine:

encoding of the simulated machine

as a binary string of 0’s and 1’s

M

31

A Turing Machine is described

with a binary string of 0’s and 1’s.

The set of Turing machines forms a language:

Each string of the language is

the binary encoding of a Turing Machine.

Therefore:

32

Language of Turing Machines

L = { 010100101,

00100100101111,

111010011110010101,

…… }

(Turing Machine 1)

(Turing Machine 2)

……

33

The Chomsky Hierarchy

34Non-recursively enumerable

Recursively-enumerable

Recursive

Context-sensitive

Context-free

Regular

The Chomsky Language Hierarchy

35

Unrestricted Grammars

Productions

vu

String of variables

and terminals

String of variables

and terminals

36

Example of unrestricted grammar

dAc

cAaB

aBcS

37

A language is recursively enumerable

if and only if it is generated by an

unrestricted grammar.

L

Theorem

38

Context-Sensitive Grammars

and |||| vu

Productions

vu

String of variables

and terminals

String of variables

and terminals

39

The language }{ nnn cba

is context-sensitive:

bbBb

bcBc

aBBa

S

aBScS

40

A language is context sensitive

if and only if

it is accepted by a Linear-Bounded automaton.

L

Theorem

41

Linear Bounded Automata (LBAs)

are the same as Turing Machines

with one difference:

The input string tape space

is the only tape space allowed to use.

42

[ ]a b c d e

Left-end

marker

Input string

Right-end

marker

Working space

in tape

All computation is done between end markers.

Linear Bounded Automaton (LBA)

43

There is a language which is context-sensitive

but not recursive.

Observation

44

Decidability

45

Consider problems with answer YES or NO.

Examples

• Does Machine have three states ?M

• Is string a binary number? w

• Does DFA accept any input? M

46

A problem is decidable if some Turing machine

solves (decides) the problem.

Decidable problems:

• Does Machine have three states ?M

• Is string a binary number? w

• Does DFA accept any input? M

47

Turing MachineInput

problem

instance

YES

NO

The Turing machine that solves a problem

answers YES or NO for each instance.

48

The machine that decides a problem:

• If the answer is YES

then halts in a yes state

• If the answer is NO

then halts in a no state

These states may not be the final states.

49

YES

NO

Turing Machine that decides a problem

YES and NO states are halting states

50

Difference between

Recursive Languages (“Acceptera”) and Decidable problems (“Avgöra”)

The YES states may not be final states.

For decidable problems:

51

Some problems are undecidable:

There is no Turing Machine that

solves all instances of the problem.

52

A famous undecidable problem:

The halting problem

53

The Halting Problem

Input: • Turing Machine M• String w

Question: Does halt on ? M w

54

Theorem

The halting problem is undecidable.

Proof

Assume to the contrary that

the halting problem is decidable.

55

There exists a Turing Machine

that solves the halting problem

H

HM

w

YES M halts on w

Mdoesn’t

halt onwNO

56

H

wwM 0q

yq

nq

Input:

initial tape contents

Encoding

of M wString

YES

NO

Construction of H

57

Construct machine H

returns YES then loop forever. HIf

returns NO then halt.HIf

58

H

wwM 0q

yq

nq NO

aq bq

H

Loop forever

YES

59

HConstruct machine

Input:

If M halts on input Mw

Then loop forever

Else halt

Mw (machine )M

60

Mw MM wwcopy

Mw H

H

61

HRun machine with input itself

Input:

If halts on input

Then loop forever

Else halt

Hw ˆ (machine )H

H Hw ˆ

62

on input H Hw ˆ

If halts then loops forever.

If doesn’t halt then halts.

:

H

H

CONTRADICTION !

H

H

63

This means that


END OF PROOF

64

Another proof of the same theorem

If the halting problem was decidable then

every recursively enumerable language

would be recursive.

65

Theorem


Proof


the halting problem is decidable.

66

There exists Turing Machine

that solves the halting problem.

H

HM

w

YES M halts on w

Mdoesn’t

halt onwNO

67

Let be a recursively enumerable language. L

Let be the Turing Machine that accepts .M L

We will prove that is also recursive: L

We will describe a Turing machine that

accepts and halts on any input.L

68

M halts on ?wYES

NOM

w

Run

with input

Mw

Hreject w

accept w

reject w

Turing Machine that accepts

and halts on any input

L

Halts on final state

Halts on non-final

state

69

Therefore L is recursive.

But there are recursively enumerable

languages which are not recursive.

Contradiction!

Since is chosen arbitrarily, we have

proven that every recursively enumerable

language is also recursive.

L

70

Therefore, the halting problem is undecidable.

END OF PROOF

71

A simple undecidable problem:

The Membership Problem

72

The Membership Problem

Input: • Turing Machine M

• String w

Question: Does accept ? M w

73

Theorem

The membership problem is undecidable.

Proof


the membership problem is decidable.

74

There exists a Turing Machine

that solves the membership problem

H

HM

w

YES M accepts w

NO M rejects w

75

Let be a recursively enumerable language. L

Let be the Turing Machine that accepts .M L

We will prove that is also recursive: L

We will describe a Turing machine that

accepts and halts on any input.L

76

M accepts ?wNO

YESM

w

Haccept w

Turing Machine that accepts

and halts on any input

L

reject w

77

Therefore, L is recursive.

But there are recursively enumerable

languages which are not recursive.

Contradiction!

Since is chosen arbitrarily, we have

proven that every recursively enumerable

language is also recursive.

L

78

Therefore, the membership problem

is undecidable.

END OF PROOF

79

Reducibility

80

Problem is reduced to problemA B

If we can solve problem then

we can solve problem .

BA

B

A

81

If is decidable then is decidable.B A

If is undecidable then is undecidable.A B

Problem is reduced to problemA B

82

Example

the halting problem

reduced to

the state-entry problem.

83

The state-entry problem

Inputs:

Question:

M•Turing Machine

•State q

•String w

Does M enter state q

on input ?w

84

Theorem

The state-entry problem is undecidable.

ProofReduce the halting problem to

the state-entry problem.

85

Suppose we have an algorithm (Turing Machine)

that solves the state-entry problem.

We will construct an algorithm

that solves the halting problem.

86

Algorithm for

state-entry

problem

M

w

q

YES

NO

entersM q

doesn’t

enterM q

Assume we have the state-entry algorithm:

87

Algorithm for

Halting problem

M

w

YES

NO

halts onM w

doesn’t

halt onM w

We want to design the halting algorithm:

88

Modify input machine M• Add new state q

• From any halting state add transitions to q

M q

halting statesSingle

halt state

M

89

M halts

M halts on state q

if and only if

90

Algorithm for halting problem

Inputs: machine and stringM w

2. Run algorithm for state-entry problem

with inputs: M wq, ,

1. Construct machine with state M q

91

Generate

M M

w

M qw

State-entry

algorithm

Halting problem algorithm

YES

NO

YES

NO

92

Since the halting problem is undecidable,

it must be that the state-entry problem

is also undecidable.

END OF PROOF

We reduced the halting problem

to the state-entry problem.

93

Another example

The halting problem

reduced to

the blank-tape halting problem.

94

The blank-tape halting problem

Input: MTuring Machine

Question: Does M halt when started with

a blank tape?

95

ProofReduce the halting problem to the

blank-tape halting problem.

Theorem

The blank-tape halting problem is undecidable.

96

Suppose we have an algorithm

for the blank-tape halting problem.

We will construct an algorithm

for the halting problem.

97

Algorithm for

blank-tape

halting problemM

YES

NO

halts on

blank tape

M

doesn’t halt

on blank tape

M

Assume we have the

blank-tape halting algorithm

98

Algorithm for

halting problem

M

w

YES

NO

halts onM w

doesn’t

halt onM w

We want to design the halting algorithm:

99

wMConstruct a new machine

• On blank tape writes w• Then continues execution like M

wM

Mthen write w

step 1 step2

if blank tape execute

with input w

100

M halts on input string

wM halts when started with blank tape.

if and only if

w

101


1. Construct wM

2. Run algorithm for

blank-tape halting problem

with input wM

Inputs: machine and stringM w

102

Generate

wMMw

Blank-tape

halting

algorithm

Halting problem algorithm

YES

NOwM

YES

NO

103


the blank-tape halting problem is

also undecidable.

END OF PROOF

We reduced the halting problem

to the blank-tape halting problem.

104

Summary of Undecidable Problems

Halting Problem

Does machine halt on input ?M w

Membership problem

Does machine accept string ?M w

Is a string member of a

recursively enumerable language ?)Lw(In other words:

105

Does machine halt when starting

on blank tape?

Blank-tape halting problem

M

State-entry Problem:

Does machine enter state

on input ?

Mw

q

106

Uncomputable Functions

107

Uncomputable Functions

A function is uncomputable if it cannot

be computed for all of its domain.

Domain Rangef

108

An uncomputable function:

)(nfmaximum number of moves until

any Turing machine with states

halts when started with the blank tape.

n

Example

109

TheoremFunction is uncomputable.)(nf

Then the blank-tape halting problem

is decidable.

ProofAssume to the contrary that

is computable.)(nf

110

Algorithm for blank-tape halting problem

Input: machine M

1. Count states of : M m

2. Compute )(mf

3. Simulate for steps

starting with empty tape

M )(mf

If halts then return YES

otherwise return NO

M

111

Therefore, the blank-tape halting

problem must be decidable.

However, we know that the blank-tape

halting problem is undecidable.

Contradiction!

112

Therefore, function is uncomputable.)(nf

END OF PROOF

113

A language L is recursively enumerable if

There is a Turing machine M accepting all

Strings w in L. This is denoted L=L(M)

A language L is recursive if there is a Turing

Machine M, that halts on any input w and deciding weather w is in L or not.

Definition

114

A language L is recursively enumerable iff it

Is generated by an unrestricted grammar.

Another definition

115

A language L is recursive iff L is recursivelyenumerable AND it’s complement L’ is recursively enumerable

Implication

116

Proof sketch (if)

Let L be a recursive language. Then there is

A Turing machine M such that running any input w on M, M will answer yes if w is in L and no if w is not in L.

Construct a machine ML accepting L by modifying M such as the yes state is a final state (all other rejecting).

Construct a machine ML’ accepting L’ by modifying M such that the no state is a final state (all other rejecting).

117

Proof sketch (only if)

Let M be a Turing machine accepting L and let M’ be a Turing machine accepting L’

Let ML be the Turing machine deciding L.

Run M and M’ in parallel on input w.

One of the machines will accept w.

Let ML answer yes if M accepts, and no if M’

Accepts. Since ML clearly can be constructed.

L is a recursive language.

118

Let ∑ = {a}. Let H be the set of all Turing

Machines with this alphabet. Note that H is

countably infinite! Thus, H = {M0,M1,M2,…}

Let L(Mi) denote the recursively enumerable

Language of Mi (that is, the set of strings that

Drives Mi into a final state.)

Not all recursive languages are recursively

enumerable

119

Let

L is recursively enumerable.

Pseudo-code:

1.Find i by counting a’s in w

2.Find Mi by the known encoding

3.Run Mi with w. Accept if Mi accepts


enumerable

)}(|{ iii MLaaL

120

Let

L’ is not recursively enumerable. Assume for contradiction it is: there is a TM M’ that

Accepts w if w is in L’.

Note: M’ has {a} as alphabet, so M’ = Mk for some integer k.


enumerable

)}(|{ iii MLaaL

121

Let Mk be the machine accepting L’. That is

L’ = L(Mk).

LetIs w in L’?


enumerable

kaw

)}(|{ iii MLaaL

122

Let Mk be the machine accepting L’. That is

L’ = L(Mk).

Contradiction!! L’ is not RE


enumerable)}(|{ i

ii MLaaL

LMLwLw

LMLwLw

k

k

)(

)(

123

L’ is not RE, but L is. According to previous

Result a language is recursive iff L and L’ is RE.

Conclusion: L is recursively enumerable

but not recursive.


enumerable

124

Rice’s Theorem

125

Non-trivial properties of

recursively enumerable languages:

any property possessed by some (not all)

recursively enumerable languages.

Definition

126

Some non-trivial properties of

recursively enumerable languages:

• is emptyL

L• is finite

L• contains two different strings

of the same length

127

Rice’s Theorem

Any non-trivial property of

a recursively enumerable language

is undecidable.

128

We will prove some non-trivial properties

without using Rice’s theorem.

129

Theorem

For any recursively enumerable language Lit is undecidable whether it is empty.

Proof

We will reduce the membership problem

to the problem of deciding whether

is empty.

L

130

Membership problem:

Does machine accept string ?wM

131

Algorithm for

empty language

problem

M

YES

NO

Assume we have the empty language algorithm:

Let be the machine that accepts M L

)(ML

)(ML

empty

not empty

LML )(

132

Algorithm for

membership

problem

M

w

YES

NO

acceptsM w

rejectsM w

We will design the membership algorithm:

133

First construct machine : wM

When enters a final state,

compare original input string with . wM

Accept if original input is

the same as .w

Construct

M

w

wM

134

Lw

)( wML is not empty

if and only if

}{)( wML w

135

Algorithm for membership problem

Inputs: machine and string M w

1. Construct wM

2. Determine if is empty )( wML

YES: then )(MLw

NO: then )(MLw

136

construct

wM

Check if

)( wML

is empty

YES

NO

M

w

NO

YES

Membership algorithm

wM

137

Since the membership problem is undecidable,

the empty language problem is

also undecidable.

END OF PROOF

We reduced the empty language problem

to the membership problem.

138

Decidability…continued…

139

Theorem

For a recursively enumerable language Lit is undecidable to determine whether

is finite. L

Proof

We will reduce the halting problem

to the finite language problem.

140

Assume we have the finite language algorithm:

Algorithm for

finite language

problem

M

YES

NO

)(ML

)(ML

finite

not finite

Let be the machine that accepts M L

LML )(

141

We will design the halting problem algorithm:

Algorithm for

Halting problem

M

w

YES

NO

halts onM w

doesn’t

halt onM w

142

First construct machine .wM

When enters a halt state,

accept any input (infinite language).

M

Initially, simulates on input . M w

Otherwise accept nothing (finite language).

143

M halts on

)( wML is not finite.

if and only if

w

144

Algorithm for halting problem:


1. Construct wM

2. Determine if is finite )( wML

YES: then doesn’t halt on M wNO: then halts on M w

145

construct

wM

Check if

)( wMLis finite

YES

NO

M

w

NO

YES

Machine for halting problem

146


the finite language problem is

also undecidable.

END OF PROOF

We reduced the finite language problem

to the halting problem.

147

Theorem

For a recursively enumerable language Lit is undecidable whether contains

two different strings of same length.

L

ProofWe will reduce the halting problem

to the two strings of equal length- problem.

148

Assume we have the two-strings algorithm:

Let be the machine that accepts M LLML )(

Algorithm for

two-strings

problem

M

YES

NO

)(ML

)(ML

contains

doesn’t

contain

two equal length strings

149

We will design the halting problem algorithm:

Algorithm for

Halting problem

M

w

YES

NO

halts onM w

doesn’t

halt onM w

150

First construct machine . wM

When enters a halt state,

accept symbols or .

M

Initially, simulates on input . M w

a b

(two equal length strings)

151

M halts on

wM

if and only if

w

accepts and a b

(two equal length strings)

152



1. Construct wM

2. Determine if accepts

two strings of equal lengthwM

YES: then halts on M w

NO: then doesn’t halt on M w

153

construct

wM

Check if)( wML

has two

equal length

strings

YES

NO

M

w

YES

NO

Machine for halting problem

154


the two strings of equal length problem is

also undecidable.

END OF PROOF

We reduced the two strings of equal length -

problem to the halting problem.

155

Rices theorem

If is a set of recursively enumerable languages containing some but not all such languages, then no TM can decide for an arbitrary Recursively enumerable language L, if L belongs to or not.

156

Example

Given a Turing Machine M, is it possible to decide weather all strings acceted by M begins and ends with the same symbol?

157

Undecidable

The problem is about a non-trivial language property. There are recursively enumerable languages with this property and there are recursively enumerable langages without this property.

158

Formally:

= { L | L is a recursively enumerable language where all strings begin and end with the same symbol }

159

Interaction: Conjectures, Results, and Myths

Dina GoldinUniv. of Connecticut, Brown University

http://www.cse.uconn.edu/~dqg

160

Fundamental Questions Underlying Theory of

Computation

What is computation?

How do we model it?

161

Shared Wisdom(from our undergraduate Theory of Computation courses)

computation: finite transformation of input to output

input: finite size (e.g. string or number)

closed system: all input available at start, all output generated at end

behavior: functions, transformation of input data to output data

Church-Turing thesis: Turing Machines capture this (algorithmic) notion of computation

Mathematical worldview: All computable problems

are function-based.

162

“The theory of computability and non-computability [is] usually referred to as the theory of recursive functions... the notion of TM has been made central in the development."

Martin Davis, Computability & Unsolvability, 1958

“Of all undergraduate CS subjects, theoretical computer science has changed the least over the decades.”

SIGACT News, March 2004

“A TM can do anything that a computer can do.”

Michael Sipser, Introduction to the Theory of Computation, 1997

The Mathematical Worldview

163

The Operating System Conundrum

Real programs, such as operating systems and word processors, often receive an unbounded amount of input over time, and never "finish" their task. Turing machines do not model such ongoing computation well…

[TM entry, Wikipedia]

If a computation does not terminate,

it’s “useless” – but aren’t OS’s

useful??

164

Rethinking Shared Wisdom:(what do computers do?)

computation: finite transformation of input to output

input: finite-size (string or number)

closed system: all input available at start, all output generated at end

behavior: functions, algorithmic transformation of input data to output data

Church-Turing thesis: Turing Machines capture this (algorithmic) notion of computation

computation: ongoing process which performs a task or delivers a service

dynamically generated stream of input tokens (requests, percepts, messages)

open system: later inputs depend on earlier outputs and vice versa (I/O entanglement, history dependence)

behavior: processes, components, control devices, reactive systems, intelligent agents

Wegner’s conjecture: Interaction is more powerful than algorithms

165

Example: Driving home from work

Algorithmic input: a description of the world (a static “map”)

Output: a sequence of pairs of #s (time-series data)- for turning the wheel- for pressing gas/break

Similar to classic AI search/planning problems.

166

But… in a real-world environment, the output depends on every grain of sand in the road (chaotic behavior).

Can we possibly have a map that’s detailed enough?

Worse yet… the domain is dynamic. The output depends on weather conditions, and on other drivers and pedestrians.

We can’t possibly be expected to predict that in advance!

Nevertheless the problem is solvable!

Google “autonomous vehicle research”

Driving home from work (cont.)

?

167

Driving home from work (cont.)

The problem is solvable interactively.

Interactive input: stream of video camera images, gathered as we are driving

Output: the desired time-series data, generated as we are driving

similar to control systems, or online computation

A paradigm shift in the conceptualization of computational problem solving.

168

• Rethinking the mathematical worldview

• Persistent Turing Machines (PTMs)

• PTM expressiveness

• Sequential Interaction – Sequential Interaction Thesis

• The Myth of the Church-Turing Thesis

– the origins of the myth

• Conclusions and future work

Outline

169

Sequential Interaction

• Sequential interactive computation:

system continuously interacts with its environment by alternately accepting an input string

and computing a corresponding output string.

• Examples:

- method invocations of an object instance in an OO language

- a C function with static variables- queries/updates to single-user databases- recurrent neural networks

- control systems- online computation- transducers- dynamic algorithms- embedded systems

170

Sequential Interaction Thesis

• Universal PTM: simulates any other PTM– Need additional input describing the PTM (only once)

• Example: simulating Answering Machine(simulate AM, will-do), (record hello, ok), (erase, done), (record John, ok),(record Hopkins, ok), (playback, John Hopkins), …

Simulation of other sequential interactive systems is analogous.

Whenever there is an effective method for performing sequential interactive computation, this computation

can be performed by a Persistent Turing Machine

171

Church-Turing Thesis Revisited

• Church-Turing Thesis:

Whenever there is an effective method for obtaining the values of a mathematical function, the function can be computed by a Turing Machine

• Common Reinterpretation (Strong Church-Turing Thesis)

A TM can do (compute) anything that a computer can do

• The equivalence of the two is a myth – the function-based behavior of algorithms does not capture all

forms of computation

– this myth has been dogmatically accepted by the CS community

• Turing himself would have denied it

– in the same paper where he introduced what we now call Turing Machines, he also introduced choice machines, as a distinct model of computation

– choice machines extend Turing Machines to interaction by allowing a human operator to make choices during the computation.

172

Origins of the Church-Turing Thesis Myth

A TM can do anything that a computer can do.

Based on several claims:1. A problem is solvable if there exists a Turing Machine

for computing it.

2. A problem is solvable if it can be specified by an algorithm.

3. Algorithms are what computers do.

Each claim is correct in isolationprovided we understand the underlying assumptions

Together, they induce an incorrect conclusion

TMs = solvable problems = algorithms = computation

173

Deconstructing the Turing Thesis Myth (1)

TMs = solvable problems

• Assumes:All computable problems are function-based.

• Reasons:

– Theory of Computation started as a field of mathematics; mathematical principles were adopted for the fundamental notions of computation, identifying computability with the computation of functions, as well as with Turing Machines.

– The batch-based modus operandi of original computers did

not lend itself to other conceptualizations of computation.

174


solvable problems = algorithms

Assumes:- Algorithmic computation is also function based;

i.e., the computational role of an algorithm is to transform input data to output data.

• Reasons: – Original (mathematical) meaning of “algorithms”

E.g. Euclid’s greatest common divisor algorithm

– Original (Knuthian) meaning of “algorithms”

“An algorithm has zero or more inputs, i.e., quantities which are given to it initially before the algorithm begins.“ [Knuth’68]

175


algorithms = computation

• Reasons: – The ACM Curriculum (1968): Adopted algorithms as the central

concept of CS without explicit agreement on the meaning of this term.

– Textbooks: When defining algorithms, the assumption of their closed function-based nature was often left implicit, if not forgotten

“An algorithm is a recipe, a set of instructions or the specifications of a process for doing something. That something is usually solving a problem of some sort.” [Rice&Rice’69]

“An algorithm is a collection of simple instructions for carrying out some task. Commonplace in everyday life, algorithms sometimes are called procedures or recipes...” [Sipser’97]

176

• Rethinking the mathematical worldview

• Persistent Turing Machines (PTMs)

• PTM expressiveness

• Sequential Interaction

• The Myth of the Church-Turing Thesis

• Conclusions and future work

Outline

177

The Shift to Interaction in CS

Computation = transforming input to output

Computation = carrying out a task over time

Logic and search in AI Intelligent agents, partially observable environments, learning

Procedure-oriented programming

Object-oriented programming

Closed systems Open systems

Compositional behavior Emergent behavior

Rule-based reasoning Simulation, control, semi-Markov processes

Algorithmic Interactive

178

The Interactive Turing Test

• From answering questions to holding discussions.• Learning from -- and adapting to -- the questioner.• “Book intelligence” vs. “street smarts”.

“It is hard to draw the line at what is intelligence and what is environmental interaction. In a sense, it does not really matter which is which, as all intelligent systems must be situated in some world or other if they are to be useful entities.” [Brooks]

179

• Many other interactive models– Reactive [MP] and embedded systems– Dataflow, I/O automata [Lynch], synchronous languages, finite/pushdown

automata over infinite words– Interaction games [Abramsky], online algorithms [Albers]– TM extensions: on-line Turing machines [Fischer], interactive Turing

machines [Goldreich]...

• Concurrency Theory– Focuses on communication (between concurrent agents/processes) rather than

computation [Milner]– Orthogonal to the theory of computation and TMs.

• What makes PTMs unique?– Provably more expressive than TMs.– Bridging the gap between concurrency theory (labeled transition systems) and

traditional TOC.

Modeling Interactive Computation: PTMs in Perspective

180

• Theory of Sequential Interactionconjecture: notions analogous to computational complexity, logic, and recursive functions can be developed for sequential interaction computation

• Multi-stream interaction– From hidden variables to hidden interfaces

conjecture: multi-stream interaction is more powerful than sequential interaction [Wegner’97]

• Formalizing indirect interaction– Interaction via persistent, observable changes to the

common environment– In contrast to direct interaction (via message passing)

conjecture: direct interaction does not capture all forms of multi-agent behaviors

Future Work: 3 conjectures

181

Referenceshttp://www.cse.uconn.edu/~dqg/papers/

[Wegner’97] Peter WegnerWhy Interaction is more Powerful than AlgorithmsCommunications of the ACM, May 1997

[EGW’04] Eugene Eberbach, Dina Goldin, Peter Wegner Turing's Ideas and Models of Computationbook chapter, in Alan Turing: Life and Legacy of a Great Thinker, Springer 2004

[I&C’04] Dina Goldin, Scott Smolka, Paul Attie, Elaine SondereggerTuring Machines, Transition Systems, and InteractionInformation & Computation Journal, 2004

[GW’04] Dina Goldin, Peter WegnerThe Church-Turing Thesis: Breaking the Mythpresented at CiE 2005, Amsterdam, June 2005 to be published in LNCS

CD5560 FABER Formal Languages, Automata and Models of Computation Lecture 13

Documents

Transcript of CD5560 FABER Formal Languages, Automata and Models of Computation Lecture 13