1

Module 5

• Topics– Proof of the existence of unsolvable problems

• Proof Technique– There are more problems/languages than there are

programs/algorithms

– Countable and uncountable infinities

2

Overview

• We will show that there are more problems than programs– Actually more problems than programs in any

computational model (programming language)

• Implication– Some problems are not solvable

3

Preliminaries

Define set of problems

Observation about programs

4

Define set of problems

• We will restrict the set of problems to be the set of language recognition problems over the alphabet {a}.

• That is– Universe: {a}*– Yes Inputs: Some language L subset of {a}*– No Inputs: {a}* - L

5

Set of Problems *• The number of distinct problems is given by the

number of languages L subset of {a}*– 2{a}* is our shorthand for this set of subset languages

• Examples of languages L subset of {a}*– 0 elements: { }

– 1 element: {/\}, {a}, {aa}, {aaa}, {aaaa}, …

– 2 elements: {/\, a}, {/\, aa}, {a, aa}, …

– Infinite # of elements: {an | n is even}, {an | n is prime}, {an | n is a perfect square}

6

Infinity and {a}*

• All strings in {a}* have finite length

• The number of strings in {a}* is infinite

• The number of languages L in 2{a}* is infinite

• The number of strings in a language L in 2{a}* may be finite or infinite

7

Define set of programs

• The set of programs we will consider are the set of legal C++ programs as defined in earlier lectures

• Key Observation– Each C++ program can be thought of as a finite

length string over alphabet P

• P = {a, …, z, A, …, Z, 0, …, 9, white space, punctuation}

8

Example *

int main(int A[], int n){ {26 characters including newline}

int i, max; {13 characters including initial tab}

{1 character: newline}

if (n < 1) {12 characters}

return (“Illegal Input”); {28 characters including 2 tabs}

max = A[0]; {13 characters}

for (i = 1; i < n; i++) {25 characters}

if (A[i] > max) {18 characters}max = A[i]; {15 characters}

return (max); {15 characters}

} {2 characters including newline}

9

Number of programs

• The set of legal C++ programs is clearly infinite

• It is also no more than |P*|

– P = {a, …, z, A, …, Z, 0, …, 9, white space, punctuation}

10

Goal

• Show that the number of languages L in 2{a}* is greater than the number of strings in P*

– P = {a, …, z, A, …, Z, 0, …, 9, white space, punctuation}

• Problem– Both are infinite

11

How do we compare the relative sizes of infinite sets?

Bijection (yes)

Proper subset (no)

12

Bijections

• Two sets have EQUAL size if there exists a bijection between them– bijection is a 1-1 and onto function between

two sets

• Examples– Set {1, 2, 3} and Set {A, B, C}– Positive even numbers and positive integers

13

Bijection Example

• Positive Integers Positive Even Integers 1 2

2 4

3 6

... ...

i 2i

… ...

14

Proper subset

• Finite sets– S1 proper subset of S2 implies S2 is strictly

bigger than S1• Example

– women proper subset of people

– number of women less than number of people

• Infinite sets– Counterexample

• even numbers and integers

15

Two sizes of infinity

Countable

Uncountable

16

Countably infinite set S *

• Definition 1– S is equal in size (bijection) to N

• N is the set of natural numbers {1, 2, 3, …}

• Definition 2 (Key property)– There exists a way to list all the elements of set S (enumerate S) such that the following is true

• Every element appears at a finite position in the infinite list

17

Uncountable infinity *

• Any set which is not countably infinite

• Examples– Set of real numbers– 2{a}*, the set of all languages L which are a

subset of {a}*

• Further gradations within this set, but we ignore them

18

Proof

19

(1) The set of all legal C++ programs is countably infinite

• Every C++ program is a finite string

• Thus, the set of all legal C++ programs is a language LC

• This language LC is a subset of P*

20

For any alphabet , * is countably infinite

• Enumeration ordering– All length 0 strings

• ||0 = 1 string:

– All length 1 strings• || strings

– All length 2 strings• ||2 strings

– …

• Thus, P* is countably infinite

21

Example with alphabet {a,b} *

• Length 0 strings– 0 and

• Length 1 strings– 1 and a, 2 and b

• Length 2 strings– 3 and aa, 4 and ab, 5 and ba, 6 and bb, ...

• Question– write a program that takes a number as input and

computes the corresponding string as output

22

(2) The set of languages in 2{a}* is uncountably infinite

• Diagonalization proof technique – “Algorithmic” proof– Typically presented as a proof by contradiction

23

Algorithm Overview *

• To prove this set is uncountably infinite, we construct an algorithm D that behaves as follows:– Input

• A countably infinite list of languages L[] subset of {a}*

– Output• A language D(L[]) which is a subset of {a}* that is not

on list L[]

– Declaration of algorithm D?

24

Visualizing D

List L[]

L[0]

L[1]

L[2]

L[3]

...

LanguageD(L[]) not inlist L[]

Algorithm D

25

Why existence of D implies result

• If the number of languages in 2{a}* is countably infinite, there exists a list L[] s.t.– L[] is complete

• it contains every language in 2{a}*

– L[] is countably infinite

• The existence of algorithm D implies that no list of languages in 2{a}* is both complete and countably infinite– Specifically, the existence of D shows that any

countably infinite list of languages is not complete

26

Visualizing One Possible L[ ] *

L[0]

L[1]

L[2]

L[3]

L[4]...

a aa aaa aaaa ...

IN ININININ

OUTIN IN INOUT

OUT OUT OUT OUT OUT

IN INOUT OUT OUT

ININOUT OUTOUT

•#Rows is countably infinite

•Given

•#Cols is countably infinite

• {a}* is countably infinite

• Consider each string to be a feature– A set contains or does not contain each string

27

Constructing D(L[ ]) *• We construct D(L[]) by using a unique feature

(string) to differentiate D(L[]) from L[i]– Typically use ith string for language L[i]– Thus the name diagonalization

L[0]

L[1]

L[2]

L[3]

L[4]...

a aa aaa aaaa ...

IN ININININ

OUTIN IN INOUT

OUT OUT OUT OUT OUT

IN INOUT OUT OUT

ININOUT OUTOUT

OUT

IN

IN

IN

OUT

D(L[])

28

Questions *

• Do we need to use the diagonal?– Every other column and every row?– Every other row and every column?

• What properties are needed to construct D(L[])?

L[0]

L[1]

L[2]

L[3]

L[4]...

a aa aaa aaaa ...

IN ININININ

OUTIN IN INOUT

OUT OUT OUT OUT OUT

IN INOUT OUT OUT

ININOUT OUTOUT

29

Visualization

Solvable Problems

The set of solvable problems is a proper subset of the set of all problems.

All problems

30

Summary

• Equal size infinite sets: bijections– Countable and uncountable infinities

• More languages than algorithms– Number of algorithms countably infinite– Number of languages uncountably infinite– Diagonalization technique

• Construct D(L[]) using infinite set of features

• The set of solvable problems is a proper subset of the set of all problems