Computer Science: An Overview Tenth Edition by J. Glenn...

Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley

Computer Science: An Overview

Tenth Edition

by

J. Glenn Brookshear

Chapter 12:

Theory of Computation

Presentation files modified by Farn Wang

1-2


Incomputability

The issue:

There are some questions with an answer

that we, human being, cannot compute.

• Also called unsolvability

• called undecidability for Boolean problems.

– with answer yes or no.

– also called decision problem

12-2

1-3

Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 12-3

Chapter 12: Theory of Computation

• 12.1 Functions and Their Computation

• 12.2 Turing Machines

• 12.3 Universal Programming Languages

• 12.4 A incomputable Function

• 12.5 Complexity of Problems

• 12.6 Public-Key Cryptography

1-4


The computatbility issue

• functions: what we have in mind.

– can be specifications.

– that is what we want to compute.

• programs:

– can be algorithms.

• must always stop with correct answers.

• finitely many simple and executable statements.

– that is what we have at hands.

Can all functions be computed with algorithms ?

12-4

1-5


Functions

Given an input, yields the correct output.

• max(a,b) : ranges of a, b ?

• sorting: input list to output list

• sum(x1x2…xn): input list to a number

• sumcheck(x1x2…xn,y): to a Boolean value.

• prime(b1b2…bn): Are the bits for a prime ?

Note that if the range of input is finite, the

function is always computable.

12-5

1-6


A simple argument for

incomputability

Some Boolean functions of finite bit

streams is incomputable.

Why ?

• There are more such functions than

programs (or algorithms).

• Programs (algorithms) are countably many.

– as many as integers, rationals, …

• Boolean functions are uncountably many.

– as many as reals. 12-6

1-7



incomputability

Programs countably many ?

Why ?

• It is just a finite number of bits or bytes.

12-7

1-8



incomputability

Boolean functions are uncountably many.

Why ?

• We prove by contradiction.

• The contradiction is established by the

famous argument of diagonalization.

12-8

1-9


Uncountability of Boolean functions

- Proof by diagonalization

input bit streams interpreted as integers.

input 0 1 2 3 4 5 6 7 …

f0 0 1 0 1 0 1 0 1 …

f1 0 0 1 0 0 1 1 0 …

f2 0 1 1 0 0 1 1 1 …

f3 1 1 0 1 0 1 1 0 …

… … …

2(7) is

true.

3(7) is

false.

Which Boolean function is missing ?

countable

sequence

of

functions.

1-10


Functions

- our research targets of computability

A correspondence between

• a collection of possible input values and

– e.g., N, Z, R, texts, tuples, trees, graphs, ….

• a collection of possible output values

– e.g, {0,1}, N, Z, R, ….

so that each possible input is assigned a

unique output value.

We assume that all functions are total.

• defined output for all input values.

1-11


Functions (continued)

• Computing a function: Determining the

output value associated with a given set of

input values

• Noncomputable function: A function

that cannot be computed by any algorithm

– Are there any ?

Note: an algorithm

• always halts with correct output values.

• consists of finite number of effective steps.

1-12


Function

- example

converting

measurements

• in yards

• into meters

1-13


Functions

- comments on the restrictions

• Not all software systems are meant to halt.

– An OS is not supposed to halt its service.

• However, we can also use similar

techniques to show the following.

– Some system calls can be implemented with

algorithms.

– Some properties of OS cannot be checked

with algorithms.

12-13

1-14


Effective calculation

- Clarification

What is a reasonable computation step ?

• Oracle ?

• Turing machines

• recursive functions

• -calculus

• Algebrae

• Logics

1-15


Turing machine

- the components finite-state

One-way infinite infinite memory

1 1 1 0 * 0 1 1 *

1-16


Turing Machine

- Operation

• Inputs at each step

– State

– Value at current tape position

• Actions at each step

– Write a value at current tape position

– Move read/write head

– Change state

1-17


Turing machine

- an example for incrementing a

value on a tape

1-18


Church-Turing Thesis

The functions that are computable by a Turing

machine are exactly the functions that can be

computed by any algorithmic means.

• History remark: Turing used “algorithmic

means” for “function” nowadays.

1-19


Church–Turing thesis

- Church’s version

Church's thesis, Church's conjecture

Every effectively calculable function (effectively

decidable predicate) is general recursive.

(Kleene 1952:300)

12-19

1-20



- Turing’s version

Turing's thesis

Turing's thesis that every function which would

naturally be regarded as computable is

computable under his definition, i.e. by one of his

machines, is equivalent to Church's thesis by

Theorem XXX.

(Kleene 1952:376)

12-20

1-21



The proof of equivalence of the following three

computational processes

• recursion,

• λ-calculus, and

• Turing machine

by Alonzo Church, Stephen Kleene and J.B.

Rosser (1934–6) and

by Alan Turing (1936–7).

12-21

http://en.wikipedia.org/wiki/Alonzo_Church

http://en.wikipedia.org/wiki/Stephen_Kleene

http://en.wikipedia.org/wiki/J.B._Rosser

http://en.wikipedia.org/wiki/J.B._Rosser

http://en.wikipedia.org/wiki/Alan_Turing

1-22



- The fundamental premise The notion of

• "effectively computable" or

• "effectively calculable"

is a vague intuitive one.

12-22

1-23



Neither thesis can be proven.

(i) "heuristic [observational, experiential]

evidence",

(ii) the "equivalence of 'diverse formulations'"

(e.g. the three computational processes) and

(iii) on an observational analysis of a human

with a pencil and paper following a set of

rules (Turing's analysis) and of a "worker" in

boxes (Emil Post's analysis 1936).

12-23

1-24


Turing machine

- as an effective computation model Based on the Church-Turing thesis,

we can use

• Turing machines, or

• any programming system that is equivalent

to Turing machine in computation power

as an effective computation model.

12-24

1-25


Universal Programming Language

A language

• with which a solution to any computable

function can be expressed

• that is equivalent in computation power to

Turing machines.

Examples:

• “Bare Bones” and

• most popular programming languages

1-26


The Bare Bones Language

• Bare Bones is a simple, yet universal

language.

– Actually, a counter machine

– a counter is an integer variable that can only

be decremented, incremented, and tested

against zero.

• 4 types of statements

– x=0;

– x++;

– x--;

– while (x!=0) {… };

1-27


The Halting Function

- Some assumptions on the encoding

• Given the encoded version of any program

M and input value v, return

– 1 if M(v) halts, or

– 0 if M(v) does not.

• We assume

– a program can be encoded as a (huge) integer

– the input parameter values can be encoded as

an integer.

1-28


Turing machine

- Halting problem and halting function • Given

– a Bare Bone program (or a Turing machine) M and

– an input values v,

does M(v) eventually halt ?

• Conceptually, we can define a halting function,

halt(M#v)

that outputs

– 1 if M(v) halts and

– 0 otherwise.

12-28

an encoding of M

followed by v.

1-29


Diagonalization arguments

- Georg Cantor (1845-1918)

• Originally for the uncountability of reals.

• First, we need to encode all reals as binary numbers. – the first bit: the sign

– the remaining even bits for positions behind the point.

– the odd bits for positions before the point.

Example:

• 6.75 = 0 0 1 1 1 1 000000000000…

• -1.33 = 1100100010001…..

+ 1 0.5 2 0.25 4

1-30


Proof by diagonalization


Diagonalization

• Proof plan: Proof by contradiction.

• Assume reals in [0,1) are countable.

reals p1 p2 p3 p4 p5 p6 p7 p8 …

r0 1 1 0 1 0 1 0 1 …

r1 0 1 1 0 0 1 1 0 …

r2 1 1 0 1 1 1 0 1 …

r3 1 1 0 1 0 1 1 0 …

… … …

Which real is missing ?

Enumeration of

reals in binary

encoding.

1-31


Continuum Hypothesis


Since R is larger than N, what is the size

(Cardinality) of R (|R|) ?

|R|=2|N|

12-31

reals p1 p2 p3 p4 p5 p6 p7 p8 … Integer set representations

r0 1 1 0 1 0 1 0 1 … {1,2,4,6,8,…}

r1 0 1 1 0 0 1 1 0 … {2,3,6,7,…}

r2 1 1 0 1 1 1 0 1 … {1,2,4,5,6,8,…}

r3 1 1 0 1 0 1 1 0 … {1,2,4,6,7,…}

… … … …

1-32




• There are different cardinalities of infinite

sets.

• Transfinite numbers

– 0, the number of integers

– i+1 =2i

• 1 is the number of reals

The hypothesis (The 1st Hilbert’s problem).

• 0 < 1 < … < k < k+1 < ……….

• There is no set S with 0 < |S| < 1.

12-32

1-33




There is no set S with 0 < |S| < 1

• The first Hilbert’s open problem.

• Cantor tried to prove it for many years in

vain.

• With the standard axioms of set theory,

– Gödel proved that it cannot be disproved

(1940).

– Cohen proved that it cannot be proved (1963).

• Still an open problem! 12-33

1-34


Intuitive argument for

incomputability.

Why some functions are incomputable as

programs ?

• the number of programs is 0

– program files are bits in the storage.

• The number of functions is 1

– f(1)=1, f(2)=0, f(3)=1, f(4)=1, ……

• So it is not possible to define a one-to-one

relation between programs and functions.

12-34

1-35


Some incomputable problems (1/2)

• The validity of 1st-order logic formula

(Hilbert’s 2nd problem)

xyz(single(x) (parent(y,x)worried(y))

• Mortal matrix problem

Given 15 33 matrices, M1, …, M15, are

there Mi1 … Mim = 0 ?

• CFL ambiguity problem

12-35

1-36


Some incomputable problems (2/2)

• Multivariable polynomial equations

(Hilbert’s 10th problem)

Incomputable for degree 4.

Incomputable for 36 real variables.

Incomputable for 11 integer variables.

12-36

1-37


Complexity of Problems

• Time Complexity: The number of instruction executions required

– Unless otherwise noted, “complexity” means “time complexity.”

• Complexity upper-bound:

– A problem is in class O(f(n)) if it can be solved by an algorithm in Q(f(n)).

• Complexity lower-bound:

– A problem is in class Q(f(n)) if the best algorithm to solve it is in class Q(f(n)).

1-38


A procedure MergeLists for merging

two lists

1-39


MergeList(h, len)

Given A[1..n],

• assume A[h,h+len-1] and

A[h+len, h+2len-1] are sorted

• sort A[h,h+len-1] and A[h+len, h+2len-1]

into A[h,h+2len-1].

12-39

1-40


MergeList(h, len)

let a = h; b = h+len-1;

for (; a< h+len && b < h+2len; ) {

if (A[a] < A[b]) { output A[a]; a++; }

else { output A[b]; b++; }

}

if (a < h+len) for (; a<h+len; a++) output A[a];

else for (; b<h+2len;b++) output A[b];

12-40

1-41


MergeList

- in the running

12-41

4 9 2 3

7 8 1 5

8 9 5 7 3 4 1 2

1-42


The merge sort algorithm

implemented as a procedure

MergeSort

merg

1-43


MergeSort(A, n)

for (i=1; i <= log(n); i++) {

k = 2^i;

for (h = 1; h <= n; h=h+2k)

A[h,h+2k-1] = MergeList(h,k);

}

complexity O(nlog n)

12-43

1-44


MergeSort

12-44

9

4

2

3

1

5

7

8

4 9

5 7

1 8

2 3

4 9 2 3

7 8 1 5

8 9 5 7 3 4 1 2

1-45


The hierarchy of problems generated

by the merge sort algorithm

1-46


Graphs of the mathematical

expressions n, lg n, n lg n, and n2

1-47


Complexities

Given input size of n bits

• of algorithms

– how many steps O(f(n)) needed to compute

f(k) with log k = n.

• of problems

– lower bound f(n): for any input of n bits, you

can not design an algorithm that runs in lower

than O(f(n)).

– upper-bound f(n): you know that there is an

algorithm that runs in O(f(n)).

12-47

1-48


A graphic summation of the problem

classification

PSPACE

EXPTIME

EXSPACE

Double EXPTIME

non-elementary

1-49


common complexity heirarchy

• O(log(n)): searching

• O(n), O(p(n)): p(n) is a polynomial of n.

– also called P, or PTIME, deterministic

• NP, nondeterministically polynomial time

• PSPACE, space(p(n))

• EXPTIME, time O(2^n)

• NEXPTIME

• EXPSPACE

• DEXPSPACE, time O(2^{2^n})

• …

• non-elementary complexity O(2^{2^{2^{…..}}})

• incomputable

12-49

1-50


A non-deterministic finite-state

machine

12-50

buck

ginger ale

coffee

buck

one event, many

choices

1-51


A non-deterministic Turing machine

12-51

1dollar

ginger ale

coffee

1dollar

1-52


Nondeterministic algorithms

• An “algorithm” whose steps may not be

uniquely and completely determined by the

process state

• Intuitively, you can guess in

nondeterministic algorithm.

• The problem is considered solved if one of

the guess solves the problem.

1-53


NP-complete problems

Given input of size n, the solution of the

problem can be guessed in f(n) steps (by a

Turing machine) where f() is a polynomial

function.

12-53

In f(n) steps to halt.

1-54


NP-complete problems

• The easiest hard problems

• including many planning problems.

– Tour planning problem with a constraint.

– Bin packing problem.

– ……

12-54

1-55


Boolean formula satisfiability problem.

- The first NP-complete problem.

Satisfiability problem of Boolean formula.

f(x,y,z) = (x y z) (x y z)

Is there valuation that satisfies f(x,y,z) ?

Why it is NP-complete ?

1. By guessing the values of x,y,z, we may

solve the problem in |f()| steps.

2. By reduction from Turing machine NP

problems.

12-55

1-56


SAT problem - Reduction from Turing machine NP problems.

Construct a formula of f(n) * f(n) * C * p

variables that encodes the f(n) tape

configurations of the computation of |f(n)|

steps.

12-56

In f(n) steps to halt.

states of control head positions

1-57



- Example

Traveling saleman: Given a map of cities,

find a tour that visit all cities exactly once

with shortest distance.

• polynomial time with nondeterministic

algorithms

• not known to be in PTIME

• an NP-complete problem!

1-58



- Example Bin Packing: Given

• a backpack of capacity B and

• n items with sizes s1, …, sn,

What is the minimum number of times that

we need to carry the backpack in order to

convey the n items ?

• polynomial time with nondeterministic

algorithms

• not known to be in PTIME

• an NP-complete problem.

1-59


P versus NP • Class P: All problems in any class Q(f(n)), where

f(n) is a polynomial

• Class NP: All problems that can be solved by a

nondeterministic algorithm in polynomial time

Nondeterministic algorithm

• an “algorithm” whose steps may not be uniquely

and completely determined by the process state

• Intuitively, you can guess in nondeterministic

algorithm.

• The problem is considered solved if one of the

guess solves the problem.

• Whether the class NP is bigger than class P is

currently unknown.

1-60


Reduction

- Establishing complexity lower-bound

To show a problem A is at least of a certain

complexity K,

• we may use another problem B of

complexity K,

• propose a construction procedure () such

that for every problem instance b of B,

– (b) is a problem instance of A, and

– (b) can be constructed in lower than K

compleixty.

12-60

1-61


Example of a reduction

From a Boolean satisfiability problem

to a vertex cover problem.

• A vertex cover of a graph is a set of vertices

such that each edge of the graph is incident to at

least one vertex of the set.

12-61

http://en.wikipedia.org/wiki/Boolean_satisfiability_problem

http://en.wikipedia.org/wiki/Vertex_cover_problem

http://en.wikipedia.org/wiki/Graph_(mathematics)

http://en.wikipedia.org/wiki/Graph_(mathematics)

1-62


Example of a reduction

From a Boolean satisfiability problem


• The problem of finding a minimum vertex cover

is a classical optimization problem.

• A typical example of an NP-hard

12-62



http://en.wikipedia.org/wiki/Optimization_problem

http://en.wikipedia.org/wiki/NP-hard




1-63


Example of a reduction From a Boolean satisfiability problem


Blue vertices form a vertex cover which corresponds to truth values.

12-63

A

A B

Theorem [Karp’72]:

There is a truth

assignment

if and only if there the

minimum vertex cover

is L+2C.

• L: # of variables

• C: # of clauses



1-64


Application of NP problems

- Public-Key Cryptography

• Key: A value used to encrypt or decrypt a

message

– Public key: Used to encrypt messages

– Private key: Used to decrypt messages

• RSA: A popular public key cryptographic

algorithm

– Relies on the (presumed) intractability of the

problem of factoring large numbers

1-65


Encrypting the Message 10111

• Encrypting keys: n = 91 and e = 5

• 10111two = 23ten

• 23e = 235 = 6,436,343

• 6,436,343 ÷ 91 has a remainder of 4

• 4ten = 100two

• Therefore, encrypted version of 10111 is

100.

1-66


Decrypting the Message 100

• Decrypting keys: d = 29, n = 91

• 100two = 4ten

• 4d = 429 = 288,230,376,151,711,744

• 288,230,376,151,711,744 ÷ 91 has a remainder of 23

• 23ten = 10111two

• Therefore, decrypted version of 100 is 10111.

1-67


Public key cryptography

1-68


Establishing an RSA public key

encryption system

Computer Science: An Overview Tenth Edition by J. Glenn...

Documents

Transcript of Computer Science: An Overview Tenth Edition by J. Glenn...