Computability

Kolmogorov-Chaitin-Solomonoff. Other topics.

Homework: Prepare presentations.

Information• Shannon definition: series of (binary) choices.

Information in measured in bits.• Computability definition: Let x be a binary string.

A minimal description of x, d(x) is the shortest string,<M,w>, where Turing Machine M on input w halts with x on the tape. The descriptive complexity (aka Kolmogorov or Kolmogorov-Chaitin complexity) is K(x) = |d(x)|– the length of this shortest string.– Note: there may be more than one.

Informally• Suppose we have a string consisting of 100

groups of '0110'. This is a description and it seems like it would be shorter than writing out the whole string.

• The formal description includes the TM (or program) that knows what to do with 100 groups of something. The [full] minimal description consists of this TM plus an encoding of 100 groups, 0110. – Program that takes description and produces string.

• This definition requires a TM for even the definition that is the whole string.

Common example

• Mandelbrot fractals are very intricate patterns and yet can be produced (re-produced?) by simple computer programs.

Claim

• There exists a constant c, such that for all x, K(x)<= |x| + c.– intuitively: take a fixed TM that halts

immediately. Then it halts with the string x on it.

• There exists a constant c, such that for all x and y, K(xy)<=2K(x) + K(y)+c

Incompressible strings

• Definitions: a string x is c-compressible if K(x)<=|x|-c.– If x is not c-compressible, x is incompressible

by c.– If x is incompressible by 1, x is

incompressible.

Incompressible strings exist!

• The number of strings of length n is greater than the number of descriptions of length less than n. So, some string of length n is not described by any description of length less than n.

K(x) is not computable!

• [and not because the definition is tied to any one of several models of computing]– See http://

en.wikipedia.org/wiki/Kolmogorov_complexity It is a Halting Problem type of proof.

Berry Paradox

• Let n be the smallest positive integer that cannot be defined in fewer than twenty English words. – oops!

• Relates also to Godel incompleteness results.

Recursion Theorem

also called fixed point theorem.• There exists a program that prints itself.• General strategy: create constant that

represents working part of program and write program that prints out constant.– See Logo example in Shai lectures.– c# example next slide:

http://igoro.com/archive/how-to-write-a-self-printing-program/

c# example

May need to force to be in two lines

class P{static void Main(){var S="class P{{static void Main(){{var S={1}{0}{1};System.Console

.Write(S,S,'{1}');}}}}";System.Console.Write(S,S,'"');}}

finish Breaking the code

• Discussion

Homework

• Prepare presentations!