Suport de Curs-AA

Algorithms & Complexity Theory

Matei PopoviciComputer Science Department

POLITEHNICA University of Bucharest

January 9, 2015

2

0.1 Introduction

0.1.1 The Apollo example

The story

On the 11th of April 1970, the Apollo 13 mission is taking o↵, with theintention of landing on the Moon. Two days into the mission, an explosionin the Service Module forces the crew to abort mission and attempt a returnto Earth. Facing extreme hardship, especially due to loss of power, the crewfinally succeeds in returning to Earth. Both the crew and the support teamfrom Space Center were forced to find fast and simple solutions to apparentlyinsurmontable problems1.

Drawing inspiration from this scenario, we consider the following inspi-rational example.

The space mission S takes o↵ having Moon as destination. Some timeafter take-o↵, S notices a malfunction in the main engine, which fails to

respond to command inputs. Trying to hack-control the engine thefollowing circuit is identified:

' ⌘ (¬A _B _D) ^ (¬B _ C _D) ^ (A _ ¬C _D) ^ (A _ ¬B¬D)^(B _ ¬C _ ¬D) ^ (¬A _ C¬D) ^ (A _B _ C) ^ (¬A _ ¬B _ ¬C)

If ' can be made to output 1, then the engine could be manually started. Showever notices that, with no apparent input for A, B, C and D, does thecircuit yield 1. S requires advice from Control, as how to proceed in this

situation.

The solution which Control must prompt is dependent on some key“background” information:

(i) The position of S allows only 5 minutes of window for a decision to bemade. After 5 minutes, a safe mission abort is no longer possible.2

(ii) Trying to find the solution “by hand” is unfeasible and likely to pro-duce errors.

1One example is the famous improvisation which made the Command Module’s squareCO2 filters operable in the Lunar Module, which required such filters of round shape.

2This is highly similar to the real situation of Apollo 13: given their position andmalfunctions made a direct abort and return to Earth impossible. Instead, their returntrajectory involved entering Moon orbit.

0.1. INTRODUCTION 3

(iii) The actual problem to be solved consists in finding a certain “input”I, which assigns to each of A . . .D a value from {0, 1}, such that thecircuit ' yields 1 (which we denote as '(I) = 1). We call this problemS(').

(iv) To solve S, one needs to find an algorithm for computing '(I), givenan input I.

Computing '(I). It is helpful to examine the structure of ' first, whichshows certain regularity. ' is of the form:

(L1 _ . . . _ L1n1) ^ . . . ^ (Lk _ . . . _ Lk

nk))

where each (Li _ . . . Lini) is a clause consisting of ni literals, and each literal

Lij is of the form X or ¬X, where X is a variable (input).Next, we identify a suitable representation of which our algorithm can

benefit. Assume I is represented as a vector, which, for convenience, isindexed by variable names instead of 0 . . . n�1 (n is the number of variables).Assume ' is represented as a matrix, where each column '[i] represents avector of literals. The value '[i,X] = 1 indicates that X appears as itselfin clause i, '[i,X] = 0 indicates that X appears in a literal ¬X, while'[i,X] = ? indicates that X is not present in clause '[i].

The algorithm to solve our problem, entitled CHECK(', I), proceeds asfollows:

a) Set v = 0

b) for i = 1, k:

re-set v = 0

for each variable X from I:

set v = v + '[i,X]� I[X]

if v = 0 stop and return 0; otherwise, continue;

c) return 1

The operation � is a customized XNOR, which behaves as follows: a� bis always 0 if either of a or b is ?. Hence, if a variable X does not appearin a clause i, then '[i,X] = ? will not influence the value of v. a� b is 1 ifboth operands are the same. Hence '[i,X]� I[X] = 1 if X occurs as itselfin clause i and is given value 1 in I, or if X occurs as ¬X in clause i and isgiven value 0 in I.

4

At the end of the inner loop, CHECK will have computed the value ofsome clause (Li _ . . . Li

ni), which is 1 if at least some literal is 1.

If, some clause has the computed value of 0, then ' is 0 for the input I.Finally, note that CHECK performs at most n ⇤ k computations, where

n is the number of variables, and k is the number of clauses in '.

Computing S(') To solve S, it is su�cient to take all possible inputs I,up to the number of variables which occur in ', and for each one, performCHECK(', I).

This can be achieved by viewing the vector I as a binary counter. Hence,the operation I++ is implemented as follows, where variables X are iteratedin some arbitrary order, but fixed with respect to the algorithm:

a) for each variable X in I:

if I[X] = 1 make I[X] = 0 and continue;

otherwise make I[X] = 1 and stop;

b) overflow;

The operation I ++ is said to overflow, if instruction b) is reached, namelyif all variables are set to 0 upon a traversal of all variables.

Now, we implement FIND('):

a) Set I to be the input where all variables are set to 0;

b) while overflow was not signalled:

if CHECK(', I)=1 then return 1

otherwise I++;

c) return 0

We now have a procedure for solving S, one which Control may use inorder to assist the Mission. However, several issues arise:

(V) is FIND(') correct, that is, bug-free, and returning an appropriateresult ?

(M) how long does FIND(') take to run, with respect to the size of theinput (i.e. the size of the matrix ') ?

(C) is FIND(') an optimal algorithm? Are there ones which perform bet-ter?

0.1. INTRODUCTION 5

Next, we discuss each issue in more detail: (V) denotes the Verificationproblem. Verification of computer programs (that is, of algorithm implemen-tations) is a essential nowadays, as more and more we rely on software toperform critical tasks: assist us in driving cars and flying planes, guidingmissiles and performing surgeries. Verification is equally important for themission.

(M) identifies a measurement problem. Namely, in what way Criticalresources are consumed by an algorithm. Traditionally, these resourcesare time, i.e. number of CPU operations and space i.e. computer memory.We note that, for our given formula, CHECK(', I) will run at most 8⇤4 = 32CPU operations, where 8 is the number of clauses, and 4 is the number ofvariables from any input. FIND(') will build at most 2n assignments I,where n is the number of variables. In our example, n = 4, thus we have 16di↵erent ways of assigning A,B,C and D to values 0, 1. For each, we runCHECK(', I), yielding a total of 16 ⇤ 32 = 512 CPU operations.

For the sake of our example, let us assume Mission’s computers ranone CPU instruction per second, which is not unlikely, given the hardwareavailable in the ’70s. Thus, in 5 minutes, we can perform: 5⇤60 = 300 CPUoperations ! Note that FIND(') does not finish in su�cient time: if Controlwould use this algorithm, it would waste Mission’s time.

Finally, (C) designates a complexity problem, one specific to Com-plexity Theory: “Do e�cient algorithms exist, for a given problem?”.There are also sub-questions which spawn from the former:

• is there a better encoding of the input and the variables from I, whichmakes the algorithm run faster?

• is there a certain machine which allows performing computations ina more e�cient (fast) way?

Unfortunately, the problem which we denoted by S, is traditionally calledSAT: i.e. the Satisfiability problem for boolean formulae. It has beenknown that e�cient algorithms to solve SAT are unlikely to exist, no matterthe machine of choice. Translated into simple terms, in the general case, wecan’t get substantially better than an exponential number of steps, withrespect to the formula input. If Control would have this knowledge, then itwill take no trouble in looking at specific algorithms. If would recognize theproblem to be impossible to solve in the given time constraints, and wouldrecommend Mission to return home.

To recap, (V), (M) and (C) play an important role in decision making,both at the software level, and beyond. Naturally hard problems (re)occur

6

if every field of science. Having at least a minimal knowledge on their natureis a critical step in attempting to solve them.

Disclamer

SAT is probably one of the most studied problems in Computer Science.SAT solvers are key components of many algorithms which solve even harderproblems: program verification, code optimisation, cryptography, only toname a few. While reading this, it is highly probable that the reader employssome technology which relies on SAT solvers, directly or indirectly.

Are these solvers e�cient, in the sense that, they run in less than expo-nential time? For the general case, the answer is no. However, these solverswill run fast enough for most formulae. This doesn’t really help Control,unless they are lucky enough to stumble upon a “nice” formula.

The intuition behind SAT solvers is that they rely on an e�cient graph-like structure to represent a formula. This structure is called an OBDD(Ordered-Binary Decision Diagram). The e�ciency of an OBDD is unfor-tunately dependent on finding a “suitable” ordering for the variables. Thelatter, in itself is a hard problem. However, there are good heuristics whichcome close. The overall result is that SAT solvers perform well in “many”cases, and the “many” is measurable with controlled precision.

0.1.2 Which is harder?

Consider the following problems:

A telecommunications company T owns radios across the country, andneeds to monitor all links, to record performance. Monitoring each radio isexpensive, thus T would like a minimal number of k radios to be monitored.

Is this possible?

An organism consists of a number of cells, and connections between cells.Some cells are good, while some are bad. The bad cells can be detected byexamining their links: they form a complete mesh: each cell is connected to

all others. Are there (k-) bad cells in the organism?

It is easy to notice that both problems can be cast into graph problems.Let G = (V,E) be an unoriented graph. If we interpret each node as a radio,and each edge as a radio link, then solving the former problem boils downto finding a subset S ✓ V such that, for all (a, b) 2 E, at least one of thefollowing holds: a 2 S, b 2 S. Hence, S “covers” all edges from E.

0.1. INTRODUCTION 7

If we interpret nodes as cells, and edges as connections between cells,then the latter problem consists in finding a subset S ✓ V (of size |S| = k)such that (a, b) 2 E for each a, b 2 S. Hence S is a clique (of size k).

One interesting question which may be raised is whether one problem isharder than the other. We note that a graph capturing the first problemcan be transformed into one capturing the latter, and vice-versa.

If we start from the “telecommunications” graph, we can build a “cell”graph by: (i) creating once cell per radio (ii) if two radios do not share alink, then the corresponding cells will share a connection. Thus, if somesubset S of radios covers all links, then all the cells corresponding to radiosoutside S must share connections, hence they are bad.

We note that the transformation can be easily adjusted to work in theother direction. Thus, if one has an algorithm to solve the Telecommuni-cations problem, then, via a transformation, we can solve the cell problem,and vice-versa.

This observation highlights several issues:

• If such a transformation exists between two problems, what does itsay about the complexity of solving the problems?

• Is it always the case that transformations are bijective?

• What are the properties of the transformation, such that it is e↵ective(can be used for problem solving)? For instance, if some problem Acan be transformed to B in exponential time3, and A can be solvedin polynomial time, then solving B via A yields an algorithm in expo-nential time.

• Can (appropriate) transformations be used to characterize problemswhich are equally hard?

In the previous section, we illustrated a problem which can be solved inexponential time. We claimed, without arguing, that this problem cannot besolved in polynomial time. Given some problem P , if we can appropriatelytransform P into the former problem, we can also argue that P cannot besolved in polynomial time. If this would be possible, an algorithm for Pcould also solve our former problem.

Developing a formal methodology from the above intuition is a criticaltool for assessing problem hardness.

3with respect to the size of the input

8

0.2 Computability Theory

0.2.1 Problems and problem instances

In the previous section, we have illustrated the problem SAT, as well as apseudocode which describes a solution in exponential time. We have seenthat such a solution is infeasible in practice, and also that no (predictible)technological advance can help. The main question we asked (and alsoanswered) is whether there exists a faster procedure to solve SAT. (Wehave conjectured that the answer is no, or, more precisely not likely.) Togeneralize a little, we are interested in the following question:

Can a given problem Q be solved in “e�cient” time?

For now, we go around the currently absent definition for “e�cient”, andnote that such a question spawns another (which is actually more straight-forward to ask):

Can a given problem Q be “solved” at all?

In this chapter, we shall focus on answering this question, first, and to doso, we need to settle the following issues:

• What exactly is a “problem”?

• What does it mean to “solve” a problem?

Definition 0.2.1.1 (Abstract problem, problem instance) A probleminstance is a mathematical object of which we ask a question and expect ananswer.

An (abstract) problem is a mapping P : I ! O where I is a set ofproblem instances of which we ask the same question, and O is a set ofanswers. P assigns to each problem instance i 2 I the answer P (i) 2 O.

It is often the case that the answers we seek are also mathematical ob-jects. For instance, the vector sorting problem must be answered by a sortedvector. However, many other problems prompt yes/no answers. WheneverO = {0, 1} we say that P is a decision problem. Many other problems canbe cast into decision problems. The vector sorting problem may be seenas a decision problem, if we simply ask whether the problem instance (i.e.the vector) is sorted. The original sorting problem and it’s decision coun-terpart may not seem equivalent in terms of hardness. For instance, sorting

0.2. COMPUTABILITY THEORY 9

is solved in polynomial time n log n (using “standard” algorithms) whichdeciding whether a vector is sorted can be done in linear time. We shallsee that, from the point of view of Complexity Theory, both problems areequally hard.

Definition 0.2.1.1 may seem abstract and unusable for the following rea-son: the set I is hard to characterize. One solution may be to assign typesto problem instances. For example graph may be a problem instance type.However, such a choice forces us to reason about problems separately, basedon the type of their problem instances. Also types themselves are an infiniteset, which is also di�cult to characterize.

Another approach is to level out problem instances, starting from thefollowing key observations: (i) each i 2 I must be, in some sense finite.For instance, vectors have a finite length, hence a finite number of elements.Graphs (of which we ask our questions) also have a finite set of nodes, hence,a finite set of edges, etc. (ii) I must be countable (but not necessarily finite).For instance, the problem P : R⇥R ! {0, 1} where P (x, y) returns 1 if x andy are equal, has no sense from the point of computability theory. Assumewe would like to answer P (⇡,

p2). Simply storing ⇡ and

p2, which takes

infinite space, is impossible on machines, and also takes us to point (i).The observations suggest that problem instances can be represented via

a finite encoding, which may be assumed to be uniform over all possiblemathematical objects we consider.

Definition 0.2.1.2 (Encoding problem instances) Let ⌃ be a finite setwhom we call alphabet. A one-letter word is a member of ⌃. A two-letterword is any member of ⌃ ⇥ ⌃ = ⌃2. For instance, if ⌃ = {a, b . . .}, then(a, a) 2 ⌃2 is a two-letter word. An i-letter word is a member of ⌃i. Wedenote by:

⌃⇤ = {✏} [ ⌃ [ ⌃2 [ . . . [ ⌃i [ . . .

the set of finite words which can be formed over ⌃. ✏ is a special word whichwe call empty word. Instead of writing, e.g. (a, b, b, a, a) for a 5-letter word,we simply write abbaa. Concatenation of two words is defined as usual.

Remark 0.2.1.1 We shall henceforth consider that a problem P : I ! Ohas the following property: if I is infinite, then I ' ⌃⇤ (I is isomorphicwith ⌃⇤). Thus, each problem instance i can be represented as a finite wordenc(i) 2 ⌃⇤, for some ⌃⇤.

We shall postpone, for now, the question of choosing the appropriate ⌃for our problem (the above remark simply states that such a ⌃ must exist).

10

Definition 0.2.1.2 and Remark 0.2.1.1 can be easily recognized from prac-tice. A programmer always employs the same predefined mechanisms of hisprogramming language (the available datatypes) to represent his programinputs. Moreover, these objects ultimately become streams of bits, whenthey are actually processed by the machine.

Making one step further, we can observe the following property of al-phabets, which conforms with (ii):

Proposition 0.2.1.1 For any finite ⌃, ⌃⇤ is infinitely countable.

Proof: We show ⌃⇤ ' N. We build a bijective function h which assigns toeach word, a unique natural. We assign 0 to ✏. Assume |⌃| = n. We assignto each one-letter word, the numbers from 1 to n. Next, we assign to eachk � 2-letter word w = w0x the number n ⇤ h(w0) + h(x). If n = 2 we easilyrecognise that each binary word is assigned to his natural equivalent. ⇤

Hence, we have the following diagram:

i 2 I $ enc(i) 2 ⌃⇤ $ h(enc(i)) 2 N

Hence, we can view a problem instance as a natural number, withoutlosing the ability to uniquely identify the instance at hand. Thus:

Definition 0.2.1.3 (Problem) A problem is a function f : N ! N. Ifsome n encodes a problem input, then f(n) encodes it’s answer. A decisionproblem is a function f : N ! {0, 1}.

To conclude: when trying to solve concrete problems, the encoding issueis fundamental, and this is dependent on the type of problem instances wetackle. From the perspective of Computability Theory which deals withproblems in general, the encoding is unessential, and can be abstractedwithout “loss of information” by a natural number.

0.2.2 Algorithms as Turing Machines

Algorithms are usually described as pseudo-code, and intended as abstrac-tions over concrete programming language operations. The level of abstrac-tion is usually unspecified rigorously, and is decided in an ad-hoc manner bythe writer. From the author’s experience, pseudo-code is often dependenton (some future) implementation, and only abstracts from language syn-tax, possibly including data initialization and subsequent handling. Thus,some pseudo-code can be easily implemented in di↵erent languages to the


extent to which the languages are the same, or at least follow the sameprogramming principles.

The above observation is not intended as a criticism towards pseudo-code and pseudo-code writing. It is indeed di�cult, for instance, to writepseudocode which does not seem vague, and which can be naturally im-plemented in an imperative language (using assignments and iterations) aswell as in a purely functional language (where iterations are possible onlythrough recursion).

As before, we require a means for leveling out di↵erent programmingstyles and programming languages, in order to come up with a uniform,straightforward and simple definition for an algorithm.

The key observation here is that programming languages, especially thenewest and most popular, are quite restrictive w.r.t. to what the program-mer can do. This may seem counter-intuitive at first. Consider typed lan-guages for instance. Enforcing each variable to have a type is obviouslya restriction, and has a definite purpose: it helps the programmer writecleaner code, and one which is less likely to crash at runtime. However, thisissue is irrelevant from the point of view of Computability Theory. If we tryto search for less restrictive languages, we find the assembly languages. Therestrictions are minimal here (as well as the “programming structure”).

The formal definition for an algorithm which we propose, can be seenas an abstract assembly language, where all technical aspects are put aside.We call such a language the Turing Machine.

Definition 0.2.2.1 (Deterministic Turing Machine) A Deterministic Tur-ing Machine (abbreviated DTM) is a tuple M = (K,F,⌃, �, s0) where:

• ⌃ = {a, b, c, . . .} is a finite set of symbols which we call alphabet;

• K is a set of states, and F ✓ K is a set of accepting/final states;

• � : K ⇥⌃ ! K ⇥⌃⇥ {L,H,R} is a transition function which assignsto each state s 2 K and c 2 ⌃ the triple �(s, c) = (s0, c0, pos).

• s0 2 K is an initial state.

The Turing Machine has a tape which contains infinite cells in both direc-tions, and on each tape cell we have a symbol from ⌃. The Turing Machinehas a tape head, which is able to read the symbol from the current cell. Also,the Turing Machine is always in a given state. Initially (before the machinehas started) the state is s0. From a given state s, the Turing Machine readsthe symbol c from the current cell, and performs a transition. The transition

12

is given by �(s, c) = (s0, c0, pos). Performing the transition means that theTM moves from state s to s0, overrides the symbol c with c0 on the tape celland: (i) if pos = L moves the tape head on the next cell to the left, (ii) ifpos = R moves the tape head on the next cell to the right and (iii) pos = Hleaves tape head on the current cell.

The Turing Machine will perform transitions according to �.Whenever the TM reaches an accepting/final state, we say it halts. If

a TM reaches a non-accepting state where no other transitions are possible,we say it clings/hangs.

• the input of a Turing Machine is a finite word which is contained inits otherwise empty tape;

• the output of a TM is the contents of the tape (not including emptycells) after the Machine has halted. We also write M(w) to refer tothe output of M , given input w.

Example 0.2.2.1 (Turing Machine) Consider the alphabet ⌃ = {#, >, 0, 1}, the set of states K = {s0, s1, s2}, the set of final states F = {s2} andthe transition function:

�(s0, 0) = (s0, 0, R) �(s0, 1) = (s0, 1, R)�(s0,#) = (s1,#, L) �(s1, 1) = (s1, 0, L)�(s1, 0) = (s2, 1, H) �(s1, >) = (s2, 1, H)

The Turing Machine M = (K,F,⌃, �, s0) reads a number encoded in binaryon the tape, and increments it by 1. The symbol # encodes the empty celltape.4 Initially, the tape head is positioned at the most significant bit of thenumber. The Machine first goes over all bits, from left to right. When thefirst empty cell is detected, the machine goes into state s1, and starts flipping1s to 0s, until the first 0 (or the initial position, marked by >) is detected.Finally, the machine places 1 on this current cell, and enters it’s final state.

The behaviour of the transition function can be more intuitively repre-sented as in Figure 2.2.1. Each node represents a state, and each edge, atransition. The label on each edge is of the form c/c0, pos where c is thesymbol read from the current tape cell, c0 is the symbol written on the cur-rent tape cell and pos is a tape head position. The label should be read as:the machine replaces c with c0 on the current cell tape and moves in thedirection indicated by pos.

4We shall use # to refer to the empty cell, throughout the text


s0start s1 s2#/#, L

c/c,R with c 2 {0, 1}

0/1, H> /1, H

1/0, L

Figure 2.2.1: The binary increment Turing Machine

Let us consider that, initially, on the tape we have >0111 — the repre-sentation of the number 7. The evolution of the tape is shown below. Eachline shows the TM configuration at step i, that is, the tape and current state,after transition i. For convenience, we have chosen to show two empty cellsin each direction, only. Also, the underline indicates the position of the tapehead.

Transition no Tape Current state0 ##>0111## s01 ##>0111## s02 ##>0111## s03 ##>0111## s04 ##>0111## s05 ##>0111## s16 ##>0110## s17 ##>0100## s18 ##>0000## s19 ##>1000## s2

In order to better understand the Turing Machine, it is useful to estab-lish some similarities with, e.g. assembly languages. As specified in Defi-nition 0.2.2.1, a Turing Machine is M specifies a clearly defined behaviour,which is actually captured by �. Thus, M is quite similar to a specific pro-gram, performing a definite task. If programs (algorithms) are abstractedby Turing Machine, then what is the abstraction for the programming lan-guage? The answer is, again, the Turing Machine. This implies that, aTuring Machine acting as a programming language, can be fed another Tur-ing Machine acting as program, and execute it.

In the following Proposition, we show how Turing Machines can be en-coded as words:

Proposition 0.2.2.1 (TMs as words) Any Turing Machine M = (K,F,⌃,�, s0) can be encoded as a word over ⌃. We write enc(M) to refer to this

14

word.

Proof:(sketch) Intuitively, we encode states and positions as integers n 2 N,transitions as pairs of integers, etc. and subsequently “convert” each integerto it’s word counterpart in ⌃⇤, cf. Proposition 0.2.1.1.

Let NonFin = |K \ F \ {s0}| be the set of non-final states, exclud-ing the initial one. We encode each state in NonFin as an integer in{1, 2, . . . , |NonFin|} and each final state as an integer in{|NonFin|+1, . . . , |NonFin|+|F |}. We encode the initial state s0 as |NonFin|+|F | + 1, and L,H,R as |NonFin| + |F | + i with i 2 {2, 3, 4}. Each integerfrom the above is represented as a word using dlog|⌃|(|NonFin| + |F | + 4)ebits.

Each transition �(s, c) = (s0, c0, pos) is encoded as:

enc(s)#c#enc(s0)#c’#enc(pos)

where enc(·) is the encoding described above. The entire � is encoded asequence of encoded transitions, separated by #. The encoding of M is

enc(M) = enc(|NonFin|)#enc(|F |)#enc(�)

⇤Thus, enc(M) is a word, which can be fed to another Turing Machine.

The latter should have the ability to execute (or to simulate) M . This isindeed possible:

Proposition 0.2.2.2 (The Universal Turing Machine) There exists aTM U which, for any TM M , and every word w 2 ⌃⇤, takes enc(M) and was input and outputs 1 whenever M(w) = 1 and 0 whenever M(w) = 0. Wecall U , the Universal Turing Machine, and say that U simulates M .

Proof: Let M be a TM and w = c1c2 . . . cn be a word which is built fromthe alphabet of M . We build the Universal Turing Machine U , as follows:

• The input of U is enc(M)#enc(s0)#c1#c2 . . . cn. Note that enc(s0)encodes the initial state of M while c1 is the first symbol from w.The portion of the tape enc(s0)#c1#c2 . . . cn will be used to mark thecurrent configuration of M , namely the current state of M (initiallys0), the contents of M ’s tape, and M ’s current head position. Moregenerally, this portion of the tape is of the form enc(si)#u#v, withu, v 2 ⌃⇤

b and si being the current state of M . The last symbol of umarks the current symbol, while v is the word which is to the left ofthe head. Initially, the current symbol is the first one, namely c1.


• U will scan the initial state of M , then it will move on the initialsymbol from w, and finally will move on the portion of enc(M) weretransitions are encoded. Once a valid transition is found, it will executeit:

1. U will change the initial state to the current one, according tothe transition;

2. U will change the original symbol in w according to the transition;

3. U will change the current symbol, according to pos, from thetransition;

• U will repeat this process until an accepting state of M is detected,or until no transition can be performed.

⇤Propositions 0.2.2.2 and 0.2.2.1 show that TMs have the capability to

characterize both algorithms, as well as the computational framework toexecute them. One question remains: what can TMs actually compute?Can they be used to sort vectors, solve SAT, etc.? The answer, which ispositive is given by the following hypothesis:

Conjecture 0.2.2.1 (Church-Turing) Any problem which can be solvedwith the Turing Machine is “ universally solvable”.

The term “universally solvable” cannot be given a precise mathematical def-inition. We only know solvability w.r.t. concrete means, e.g. computers andprogramming languages, etc. It can be (an has been) shown that the Tur-ing Machine can solve any problem which known programming languagescan solve.5 The Turing Machine, in itself, describes a model of computationbased on side-e↵ects: each transition may modify the tape in some way.Computation can be described di↵erently, for instance: as function applica-tion, or as term rewriting. However, all other known computational modelsare equivalent to the Turing Machine, in the sense that they solve preciselythe same problems.

This observation prompted the aforementioned conjecture. It is stronglybelieved to hold (as evidence suggests), but it cannot be formally proved.

5To be fair to the TM, one would formulate this statement as: ”all programminglanguages are Turing-complete, i.e. they can solve everything the TM can solve”

16

0.2.3 Decidability

The existence of the Universal Turing Machine (U) inevitably leads to inter-esting questions. Assume M is a Turing Machine and w is a word. We usethe following convention: enc(M) w in order to represent the input of theU . Thus, U expects the encoding of a TM, followed by the special symbol ,and M ’s input w. (?) Does U halt for all inputs? If the answer is positive,then U can be used to tell whether any machine halts, for a given input.

We already have some reasons to believe we cannot answer positively to(?), if we examine the proof of Proposition 0.2.2.2. Actually (?) is a decisionproblem, one that is quite interesting and useful.

As before, we try to lift our setting to a more general one: Can anyproblem be solved by some Turing Machine. The following propositionsindicate that it’s not likely the case:

Proposition 0.2.3.1 The set T M of Turing Machines is countably infi-nite.

Proof: The proof follows immediately from Proposition 0.2.2.1. Any TuringMachine can be uniquely encoded by a string, hence the set of Turing Ma-chines is isomorphic to a subset of ⌃⇤, which in turn is countably infinite,since ⌃⇤ is countably infinite, for any ⌃. ⇤

Proposition 0.2.3.2 The set Hom(N,N) of functions f : N ! N is un-countably infinite.

Proof: It is su�cient to show that Hom(N, {0, 1}) is uncountably infinite.We build a proof by contraposition. We assume Hom(N, {0, 1}) is countablyinfinite. Hence, each natural number n 2 N corresponds to a function fn 2Hom(N, {0, 1}). We build a matrix as follows: Columns describe functionsfn : n 2 N. Rows describe inputs k 2 N. Each matrix content mi,j is thevalue of fj(i) (hence, the expected output for input i, from function fj).

In the Figure 0.2.3, we have illustrated our matrix. We now devise aproblem f⇤ as follows:

f⇤(x) =

⇢1 i↵ fx(x) = 00 i↵ fx(x) = 1

Since f⇤ 2 Hom(N, {0, 1}) it must also have a number assigned to it: f⇤ =f↵ for some ↵ 2 N. Then f⇤(↵) = 1 if f↵(↵) = 0. But f↵(↵) = f⇤(↵).Contradiction. On the other hand f⇤(↵) = 0 if f↵(↵) = 1. As before weobtain a contradiction. ⇤


f0 f1 f2 . . . fn . . .0 1 1 0 . . . 0 . . .1 0 1 1 . . . 0 . . .2 1 0 1 . . . 1 . . .. . . . . . . . . . . . . . . . . . . . .n 1 1 0 . . . 1 . . .. . . . . . . . . . . . . . . . . . . . .

Figure 2.3.2: An example of the matrix of the proof of Proposition 0.2.3.2.The value mi,j have been filled out purely for the illustration.

Propositions 0.2.3.2 and 0.2.3.1 tell us that there are infinitely morefunctions (decision problems) what means of computing them (Turing Ma-chines).

Our next step is to look at solvable and unsolvable problems, and devisea method for separating the first from the latter. In other words, we arelooking for a tool which allows us to identify those problems which aresolvable, and those which are not.

We start by observing that Turing Machines may never halt. We writeM(w) = ? to designate that M loops infinitely for input w. Also, wewrite nw 2 N to refer to the number which corresponds to w, according toProposition 0.2.1.1. Next, we refine the notion of problem solving :

Definition 0.2.3.1 (Decision, acceptance) Let M be a Turing Machineand f 2 Hom(N, {0, 1}).We say that:

• M decides f , i↵ for all n 2 N: M(w) = 1 whenever f(nw) = 1 andM(w) = 0 whenever f(nw) = 0.

• M accepts f i↵ for all n 2 N: M(w) = 1 i↵ f(nw) = 1, and M(w) =? i↵ f(n) = 0.

Note that, in contrast with acceptance, decision is, intuitively, a strongermeans of computing a function (i.e. solving a problem). In the latter case,the TM at hand can provide both a yes and a no answer to any probleminstance, while in the former, the TM can only provide an answer of yes. Ifthe answer to the problem instance at hand is no, the TM will not halt.

Based on the two types of problem solving, we can classify problems(functions) as follows:

Definition 0.2.3.2 Let f 2 Hom(N, {0, 1}) be a decision problem.

18

• f is recursive (decidable) i↵ there exists a TM M which decides f .The set of recursive functions is

R = {f 2 Hom(N, {0, 1}) | f is recursive }

• f is recursively enumerable ( semi-decidable) i↵ there exists a TMM which accepts f . The set of recursive-enumerable functions is

RE = {f 2 Hom(N, {0, 1}) | f is recursively-enumerable }

Now, let us turn our attention to question (?), which we shall formulateas a problem:

fh(nenc(M) w) =

⇢1 i↵ M(w) halts0 i↵ M(w) = ?

Hence, the input of f is a natural number which encodes a Turing Ma-chine M and an input word w. The first question we ask, is whether fh 2 R.

Proposition 0.2.3.3 fh 62 R.

Proof: Assume fh 2 R and denote by Mh the Turing Machine which decidesfh. We build the Turing Machine D, as follows:

D(enc(M)) =

⇢? i↵ Mh(enc(M) enc(M)) = 11 i↵ Mh(enc(M) enc(M)) = 0

The existence of the Universal Turing Machine guarantees that D canindeed be built, since D simulates Mh. We note that Mh(enc(M) enc(M))decides if the TM M halts with “itself ” as input (namely enc(M)).

Assume D(enc(D)) = 1. Hence Mh(enc(D), enc(D)) = 0, that is, ma-chine D does not halt for input enc(D). Hence D(enc(D)) = ?. Contradic-tion.

Assume D(enc(D)) = ?. Hence Mh(enc(D), enc(D)) = 1, and thusD(enc(D)) halts. Contradiction. ⇤

We note that the construction of D mimics the technique which weapplied for the proof of Proposition 0.2.3.2, which is called diagonalization

Exercise 0.2.3.1 Apply the diagonalization technique from the proof ofProposition 0.2.3.2, in order to prove Proposition 0.2.3.3.

Proposition 0.2.3.4 fh 2 RE.


Proof: We build a Turing Machine Mh which accepts fh. Essentially, Mh isthe Universal Turing Machine. Mh(enc(M) w) simulates M , and if M(w)halts, then it outputs 1. If M(w) does not halt, Mh(enc(M) w) = ?. ⇤

Propositions 0.2.3.3 and 0.2.3.4 produce a classification for fh. The ques-tion which we shall answer, is how to classify any problem f , by establishingmembership in R and RE, respectively. We start with a simple proposition:

Proposition 0.2.3.5 R ( RE.

Proof: R ✓ RE is straightforward from Definition 0.2.3.2. Let f 2 R, andMf be the TM which decidesMf . We build the TMM 0 such thatM 0(w) = 1i↵ Mf (w) = 1 and M 0(w) = ? i↵ Mf (w) = 0. M 0 simulates M but entersinto an infinite loop whenever Mf (w) = 0. M 0 accepts f hence f 2 RE.

R 6= RE has already been shown by Propositions 0.2.3.3 and 0.2.3.4.fh 2 RE but fh 62 R. ⇤

Thus, R and RE should be interpreted as a “scale” for solvability: Rmembership is complete solvability, RE membership is partial solvability,while non-membership in RE is “complete” unsolvability.

Remark 0.2.3.1 We note that R and RE are not the only sets of functionswhich are used in Computability Theory. It has been shown that there are“ degrees” of unsolvability, of “ higher level” than R and RE. These degreesare intuitively obtained as follows: We assume we live in a world where fhis decidable (recursive). Now, as before, we ask which problems are recursiveand which are recursively-enumerable. It turns out that, also in this idealcase, there still exist recursive and recursively-enumerable problems, as wellas some which are neither. This could be imagined as “ undecidability level1”. Now, we take some problem which is in RE on level 1, and repeat thesame assumption: it is decidable. Again, under this assumption, we findproblems in R, RE and outside the two, which form up “ undecidabilitylevel 2”. This process can be repeated ad infinitum.

Returning to our simpler classification, we must observe an interestingfeature of recursively-enumerable functions, which is also the reason theyare called this way.

Proposition 0.2.3.6 A function f 2 Hom(N, {0, 1}) is recursively enu-merable i↵ there exists a Turing Machine which can enumerate/generate

all elements in Af = {w 2 N | f(nw) = 1}. Intuitively, Af is the set ofinputs of f for which the answer at hand is yes.

20

Proof: =) Suppose f is recursively-enumerable and M accepts f . Wewrite wi to refer to the ith word from ⌃⇤. We specify the TM generatingAf by the following pseudocode:

Algorithm 1: GEN()

static Af = ;;k = 0;while True do

for 0 i k dorun M(wi);if M(wi) halts before k steps and i 62 Af then

Af = Af [ {wi};return wi

end

endk = k + 1;

end

The value of k from the for has a two-fold usage. First, it is used toexplore all inputs wi : 0 i k. Second, it is used as a time-limit for M .For each wi we run M(wi), for precisely k steps. If M(wi) = 1 in at mostk steps, then wi is added to Af , and then returned (written on the tape).Also, wi is stored for a future execution of GEN. If M(wi) = 1 for some wi,then there must exist a k : k � i such that M(wi) halts after k steps. Thus,such a k will eventually be reached.

(= Assume we have the Turing Machine GEN which generates Af .We construct a Turing Machine M which accepts f . M works as follows:

Algorithm 2: M(w)

Af = ;, n = 0;while w 62 Af do

w = GEN();Af = Af [ {w}

endreturn 1

M simply uses GEN to generate elements in Af . If w 2 Af , if willeventually be generated, and M will output 1. Otherwise M will loop.Thus M accepts f . ⇤

Proposition 0.2.3.6 is useful since, in many cases, it is easier to find agenerator for f , instead of a Turing Machine which accepts f .


Finally, in what follows, we shall take a few decision problems, and applya reduction technique, in order to prove they are not decidable.

Halting on all inputs Let:

fall(nenc(M)) =

⇢1 i↵ M halts for all inputs0 otherwise

The technique we use to show fall 62 R is called a reduction (from fh). Itproceeds as follows. We assume fall 2 R. Starting from the TM Mall whichdecides fall we built a TM which decides fh. Thus, if fall is decidable thenfh is decidable, which leads to contradiction.

First, for each fixed TM M and fixed input w 2 ⌃⇤, we build the TM⇧M,w(!) = “Replace ! by w and then simulate M(w)”. It is easy to seethat (8! 2 ⌃⇤ : ⇧M,w(!) halts) i↵ M(w) halts. Now, we build the TM Mh

which decides fh. The input of Mh is enc(M) w. We construct ⇧M,w andrun Mall(enc(⇧M,w)). By assumption Mall must always halt. If the outputis 1, then ⇧M,w(!) halts for all inputs, hence M(w) halts. We output 1. Ifthe output is 0, then ⇧M,w(!) does not halt for all inputs, hence M(w) doesnot halt. We output 0.

We have built a reduction from fall to fh: Using the TM which de-cides fall we have constructed a machine which decides fh. Since fh is notrecursive, we obtain a contradiction.

Definition 0.2.3.3 (Turing-reducibility) Let fA, fB 2 Hom(N, {0, 1}).We say fA is Turing reducible to fB, and write fA T fB i↵ there ex-ists a decidable transformation T 2 Hom(N,N) such that fA(n) = 1 i↵fB(T (n)) = 1.

Remark 0.2.3.2 (Reducibility) We note that the transformation T mustbe computable, in the sense that a Turing Machine should be able to com-pute T , for any possible valid input.When proving fhalt T fall, we havetaken nenc(M) w — an instance of fh and shown that it could be trans-formed into nenc(⇧M,w) — an instance of fall, such that fh(nenc(M) w) = 1i↵ fall(nenc(⇧M,w)) = 1. A Turing Machine can easily perform the transfor-mation of enc(M) w to enc(⇧M,w) since it involves adding some states andtransitions which precede the start-state of M , hence T is computable.

Halting on 111

f111(nenc(M)) =

⇢1 i↵ M(111) halts0 otherwise

22

We reduce fh to f111. Assume M111 decides f111. Given a Turing Ma-chine M and word w, we construct the machine:

⇧M,w(!) = if ! = 111 then M(w) else loop

We observe that (i) the transformation from enc(M) w to enc(⇧M,w) isdecidable since it involves adding precisely three states to M : these statescheck the input !, and if it is 111 — replace it with w and run M ; (ii)M111(⇧M,w) = 1 i↵ M(w) halts. The reduction is complete. f111 62 R.

Halting on some input We define:

fany(nenc(M)) =

⇢1 i↵ M(w) halts for some w 2 ⌃⇤

0 otherwise

We reduce f111 to fany. We assume fany is decided by Many. We con-struct:

⇧M (!) = Replace ! by 111 and M(111)

Now, Many(enc(⇧M )) = 1 i↵ M(111) = 1, hence we can use Many to builda machine which decides f111. Contradiction fany 62 R.

Machine halt equivalence We define:

feq(nenc(M1) enc(M2)) =

⇢1 for all w 2 ⌃⇤: M1(w) halts i↵ M2(w) halts0 otherwise

We reduce fall to feq. Let Mtriv be a one-state Turing Machine whichhalts on every input, and Meq be the Turing Machine which decides feq.Then Meq(enc(M) enc(Mtriv) = 1 i↵ M halts on all inputs. We have shownwe can useMeq in order to build a machine which decides fall. Contradiction.feq 62 R.

So far, we have used reductions in order to establish problem non-membership in R. There are other properties of R and RE which can be ofuse for this task. First we define:

Definition 0.2.3.4 (Complement of a problem) Let f 2 Hom(N, {0, 1}).We denote by f the problem:

f(n) =

⇢1 i↵ f(n) = 00 i↵ f(n) = 1

We call f the complement of f .


For instance, the complement of fh is the problem which asks if a Turing

Machine M does not halt for input w. We also note that f = f .Next, we define the class:

coRE = {f 2 Hom(N, {0, 1}) | f 2 RE}

coRE contains the set of all problems whose complement is in RE.We establish that:

Proposition 0.2.3.7 RE \ coRE = R.

Proof: Assume f 2 RE \ coRE. Hence, there exists a Turing Machine Mwhich accepts f and a Turing Machine M which accepts f . We build theTuring Machine:

M⇤(w) = for i 2 N : run M(w) for i steps.If M(w) = 1, return 1. Otherwise:run M(w) for i steps.If M(w) = 1, return 0.

Since M and M will always halt when expected result is 1, they can beused together to decide f . Hence f 2 R. ⇤

Proposition 0.2.3.8 f 2 R i↵ f 2 R.

Proof: The proposition follows immediately since the Turing Machine whichdecides f can be used to decide f , by simply switching it’s output from 0 to1 and 1 to 0. The same holds for the other direction. ⇤

We conclude this chapter with a very powerful result which states thatan category/type of problems does not belong in R.

Theorem 0.2.3.1 (Rice) Let C ✓ RE. Given a Turing Machine M , weask: “The problem accepted by M is in C?. Answering this question is notin R.

Proof: We consider that the trivial problem f(n) = 0 is not in C. Since C isnon-empty, suppose f⇤ 2 C, and since f⇤ is recursively-enumerable, let M⇤

be the Turing Machine which accepts f⇤.We apply a reduction from a variant of f111, namely fx. fx asks if a

Turing Machine halts for input x. We assume we can decide the membershipf 2 C by some Turing Machine. Based on the latter, we construct a Turing

24

Machine which decides fx (i.e. solves the halting problem for a particularinput). Let Mx be the Turing Machine which accepts fx.

Let:⇧w(!) = if Mx(w) then M⇤(!) else loop.

If f⇧w is the problem accepted by ⇧w, we show that:

f⇧w 2 C i↵ Mx(w) halts

()). Suppose f⇧w 2 C. Then ⇧w(!) cannot loop for every input ! 2 ⌃⇤. Ifthere were so, then f⇧w would be the trivial function always returning 0 forany input, which we have assumed is not in C. Thus, Mx(w) halts.((). SupposeMx(w) halts. Then the behaviour of ⇧w(!) is precisely that ofM⇤(!). ⇧w(!) will return 1 whenever M⇤(!) will return 1 and ⇧w(!) = ?whenever M⇤(!) = ?. Since f⇤ 2 C, then also f⇧w 2 C. ⇤

In Theorem 0.2.3.1, the set C should be interpreted as a property ofproblems, and subsequently of the Turing Machines which accept them.Checking if some Turing Machine M⇤ satisfies the given property is unde-cidable. Consider the property informally described as: The set of TuringMachines(/computer programs) that behave as viruses. The ability of decid-ing whether a Turing Machine behaves as a virus (i.e. belongs to the formerset) is undecidable, via Rice’s Theorem.

0.3 Complexity Theory

0.3.1 Measuring time and space

In Computability Theory, we have classified problems (e.g. in classes R andRE) based on Turing Machine’s ability to decide/accept them.

In order to classify problems based on hardness, we need to account forthe number of steps (time) and tape cells which are employed by a TuringMachine.

The amount of spent resources (time/space) by a Turing Machine Mmay be expressed as functions:

TM ,SM : ⌃⇤ ! N

where TM (w) (resp. SM (w)) is the number of steps performed (resp. tapecells used) by M , when running on input w.

This definition su↵ers from un-necessary overhead, which makes timeand space analysis di�cult. We formulate some examples to illustrate whythis is the case:

0.3. COMPLEXITY THEORY 25

Alg(n)while n < 100

n = n+ 1return 1

We note that Alg runs 100 steps for n = 0 while only one step, forn � 100. However, in practice, it is often considered that each input is aslikely to occur as any other6. For this reason, we shall adopt the followingconvention: (?) We allways consider the most expensive/defavourable case,given inputs of a certain type. In our previous example, we consider therunning time of Alg as being 100, since this is the most expensive case.

Consider the following example:

Sum(v, n)s = 0, i = 0while i < ns = s+ v[i]i = i+ 1

return s

Unlike Alg, Sum does not have a universal upper limit on it’s runningtime. The number of steps Sum executes, depends on the number of vari-ables from v, namely n, and is equal to 2n+ 3, if we consider each variableinitialisation and the return statements, as computing steps. Thus, we ob-serve that (??) The running time (resp. consumed space) of an algorithmwill grow as the size of the input grows.

We can now merge (?) and (??) into a definition:

Definition 0.3.1.1 (Running time of a TM) The running time of a Tur-ing Machine M is given by TM : N ! N i↵:

8w 2 ⌃⇤ : the nb. of transitions performed by M is at most TM (|w|)

Remark 0.3.1.1 (Consumed space for a TM) A naive definition for con-sumed space of a Turing Machine would state that SM (|w|) is the number

6This is not often the case. There are numerous algorithms which rely on some prob-ability that the input is of some particular type. For instance, e�cient SAT solvers relyon a particular ordering of variables, when interpretations are generated and verified. Oncertain orderings and certain formulae, the algorithm runs in polynomial time. The keyto the e�ciency of SAT solvers is that programmers estimate and e�cient ordering, basedon some expectancy from the input formula. The algorithm may be exponentially costlyfor some formulae, but run in close-to-polynomial time, for most of the inputs.

26

of tape cells which M employs. This definition is imprecise. Consider theTuring Machine which receives a binary word as input and compute whetherthe word is a power of 2n. Asside from reading it’s input, the machine con-sumes no space. Thus, we might refine our definition into: “SM (|w|)” is thenumber of tape writes which M performs”. This definition is also imprecise.Consider the binary counter Turing Machine from the former chapter. Itperforms a number of writes proportional to the number of consecutive 1’sfound at the end of the string. However, the counter does not use additionalspace, but only makes processing on the input.

Thus, the consumed space SM (|w|) is the number of written cells exceptfrom the input. Consider a Turing Machine which receives n numbersencoded as binary words, each having at most 4 bits, and which computesthe sum of the numbers, modulo 24. Apart from reading the 4⇤n bits and n�1word separators, the machine employs another 4 cells to hold a temporarysum. Thus, the consumed space for this machine is 47.

A formal definition for consumed space of a TM is outside the scope ofthis course, since it involves multi-tape Turing Machines. The basic idea isto separate the input from the rest of the space used for computation.

Thus, when assesing the consumed space of an algorithm, we shall neveraccount for the consumed space by the input.

Recall that, as in Computability Theory, our primary agenda is to pro-duce a classification of problems. To this end, it makes sense to first intro-duce a classification of Turing Machine running times.

Asymptotic notations

Remark 0.3.1.2 (Running times vs. arbitrary functions) In the pre-vious section, we have defined running times of a Turing Machine as func-tions: T : N ! N, and we have seen that they are often monotonicallyincreasing (n m =) T (n) T (m)). While monotonicity is commonamong the running times of conventional algorithms, it is not hard to findexamples (more-or-less realistic) where it does not hold. For instance, analgorithm may simply return, if it’s input exceeds a given size. Thus, weshall not, in general, assume that running times are monotonic.

Furthermore, we shall extend our classification to arbitrary functionsf : R ! R, since there is no technical reason to consider only functions overnaturals. In support for this, we shall also add that asymptotic notations are

7We can also build another machine which simply uses the first number to hold thetemporary sum, and this use no additional space.


useful in other fields outside complexity theory, where the assumption thatfunctions are defined over natural numbers only is not justified.

Definition 0.3.1.2 (⇥ (theta) notation) Let g : R ! R. Then ⇥(g(n))is the class of functions:

⇥(g(n)) = {f : R ! R | 9c1, c2 2 R+

9n0 2 N 8n � n0, c1g(n) f(n) c2g(n)}

Thus, ⇥(f(n)) is the class of all functions with the same asymptoticgrowth as f(n). We can easily observe that, for all continuous g, f 2Hom(R,R) such that g 2 ⇥(f(n)), we have limn!1

f(n)g(n) = c, where c 6= 0.

There is an infinite number of classes ⇥(f(n)), one for each function f .However, if g(n) 2 ⇥(f(n)), then ⇥(g(n)) = ⇥(f(n)).

It makes sense to consider classes which describe functions with infe-rior/superior asymptotic growth:

Definition 0.3.1.3 (O,⌦ notations) Let f : R ! R. Then:

O(g(n)) = {f : R ! R | 9c 2 R+

9n0 2 N 8n � n0, 0 f(n) c ⇤ g(n)}

⌦(g(n)) = {f : R ! R | 9c 2 R+

9n0 2 N 8n � n0, 0 c ⇤ g(n) f(n)}

Note that g 2 O(f(n)) =) O(g(n)) ✓ O(f(n)), while g 2 ⌦(f(n)) =)⌦(g(n)) ✓ ⌦(g(n)). Finally, ⌦(f(n)) \ O(f(n)) = ⇥(f(n)). Each of theabove propositions can be easily proved using the respective definitions ofthe notations.

O and ⌦ o↵er relaxed bounds for asymptotic function growth. Thus,g 2 O(f(n)) should be read as:The function g grows asymptotically at mostas much as f . It makes sense to also consider tight bounds:

Definition 0.3.1.4 (o,! notations)

o(g(n)) = {f : R ! R | 8c 2 R+

9n0 2 N 8n � n0, 0 f(n) c ⇤ g(n)}

!(g(n)) = {f : R ! R | 8c 2 R+

9n0 2 N 8n � n0, 0 c ⇤ g(n) f(n)}

28

Thus, g 2 o(f(n)) should be read: g grows assymptotically strictly lessthan f . We have o(f(n)) \ !(f(n)) = ; O(f(n)) \ ⌦(f(n)) = ⇥(f(n)) and!(f(n)) [⇥(f(n)) = ⌦(f(n)).

Exercise 0.3.1.1

If f(n) 2 ⌦(n2) and g(n) 2 O(n3) then f(n)/g(n) 2 . . .If f(n) 2 o(n2) and g(n) 2 ⇥(n3) then f(n) · g(n) 2 . . .If f(n) 2 ⇥(n3) and g(n) 2 o(n2) then f(n) \ g(n) 2 . . .

Exercise 0.3.1.2 Prove or disprove the following implications:

f(n) = O(log n) ) 2f(n) = O(n)f(n) = O(n2) and g(n) = O(n) ) f(g(n)) = O(n3)f(n) = O(n) and g(n) = 1 +

pf(n) ) g(n) = ⌦(log n)

Syntactic sugars

This section follows closely Lecture 2 from [1]. Quite often, asymptoticnotations are used to refer to arbitrary functions having certain propertiesrelated to their order of growth. For instance, in:

df(x)e = f(x) +O(1)

applying “rounding” to f(x), may be expressed as the original f(x) to whichwe add a function bounded by a constant. Similarly:

1

1� x= 1 + x+ x2 + x3 + ⌦(x4), for � 1 < x < 1

The above notation allows us to “formally disregard” the terms from theexpansion, by replacing them with an asymptotic notation which charac-terises their order of growth. One should make a distinction between theusage of asymptotic notations in arithmetic expressions, such as the onespreviously illustrated, and equations. Consider the following example:

f(x) = O(1/x)

which should be read: there exists a function h 2 O(1/x) such that f(x) =h(x). Similarly:

f(x) = O(log x) +O(1/x)

should be read: there exist functions h 2 O(1/x) and w 2 O(log x) such thatf(x) = w(x) + h(x). In equations such as:

O(x) = O(log x) +O(1/x)


the equality is not symmetric, and should be read from left to right: forany function f 2 O(x), there exist functions h 2 O(1/x) and w 2 O(log x)such that f(x) = w(x) + h(x). In order to avoid mistakes, the followingalgorithmic rule should be applied. When reading an equation of the form:

left = right

• each occurrence of an asymptotic notation in left should be replaced byan universally quantified function belonging to the correspondingclass.

• each occurrence of an asymptotic notation in right should be replacedby an existentially quantified function from the corresponding class.

0.3.2 Running time in complexity theory

Using asymptotic notations, we can distinguish between running (of algo-rithms) with di↵erent asymptotic growths. Experience has shown that, it isunfeasible to develop a theory which uses asymptotic notations in order toclassify problems, based on their di�culty. Thus, in complexity theory, wemake an even stronger assumption: the exponent of a polynomial functionis un-important. Recall that, with asymptotic notations, we do not di↵er-entiate between n2 and 2n2+n+1 (and denote either of the two by ⇥(n2)).In Complexity Theory, we do not distinguish between, e.g. n2 and n3, andthus, we write nO(1), thus refering to a polynomial of arbitrary degree.

Before introducing a classification of problems, there is a question whichmust be addressed: How does the encoding of a problem instance a↵ect therunning time of the subsequent algorithm?

To see why this issue is important, consider the encoding of numbersusing a single digit. (e.g. IIIIIIII encodes the number 8). A TuringMachine M which starts with the tape:

> I I I I I I # #

and increments the represented number by shifting the head to the firstempty cell, where it places I, will perform a number steps which is linearwith respect to the size of the input. Thus, the running time of M is O(n)where n = |w|, and w is M ’s input.

The Turing Machine which uses the binary alphabet, and encodes num-bers as binary words, will also run in linear time w.r.t. the size of the input,but in this case, there is an exponential gap between the two representations.

30

The representation of a natural x consumes n = dlog xe cells in the secondmachine, and x cells, in the first. Note that x = 2n.

This is one of the rare cases [2] where a bad choice of a representationmay lead to an exponential increase in the number of steps. In what follows,we assume problem instances are encoded in some default way: e.g. graphsare represented as adjacency matrices or as adjacency lists, etc. When ap-propriate representations are chosen, the computational gap between themis at most polynomial. As an example, let us compare a matrix representa-tion of a graph, with that of adjacency lists. Assume the graph is directed,contains n nodes and only one edge (u, v). The matrix representation willconsume n2 positions (out of which n2�1 are equal to 0), while the list rep-resentation will consume only one position (corresponding unique elementfrom the adjacency list of u). However, the gap between the two represen-tations is polynomial. Thus, from the point of view of Complexity Theory,it is irrellevant if we chose matrices or adjacency lists to represent graphs.

This observation is highlighted by the following Proposition:

Proposition 0.3.2.1 (The encoding does not matter) Let f : N !{0, 1} be a problem which is decided by a Turing Machine M in time T .M is defined over alphabet ⌃. Then, there exists a Turing Machine M 0 —defined over alphabet ⌃0 = {0, 1,#, >}, which decides f and runs in timeO(log |⌃|) ⇤ T (n).

Proof:(sketch) We build M 0 = (K 0, F 0,⌃0, �0, s00), from M as follows:

• ⌃0 = {0, 1,#, >}. We encode each symbol di↵erent from # (the emptycell) and > (the marker symbol of the beginning of the input), as aword w 2 ⌃0 with |w| = dlog |⌃|e. We use k to refer to the length |w|of the word w. We write enc⌃0(x), with x 2 ⌃, to refer to the encodingof symbol x 2 ⌃.

• For each state s 2 K, we build 2k+1 states q1, . . . q2k+1 2 K 0, organizedas a full binary tree of height k. The purpose of the tree is to recognizethe word enc⌃0(x) of length k from the tape. Thus, the unique stateat the root of the tree, namely q1, is responsible for recognising thefirst bit. If it is 0, M 0 will transition to q2 and if it is 1, to q3. q2 andq3 must each recognize the second bit. After their transitions, we shallbe in one of the states q4 to q8, which give us information about thefirst two bits of the word. The states from level i recognize the firsti bits of the encoded symbol enc⌃0(x). The states from the last levelare 2k in number, and recognize the last bit of the encoded symbol


enc⌃0(x). Thus, each of the 2k leaf-states in the tree corresponds toone possible symbol x 2 ⌃ which is encoded as enc⌃0(x). We connectall 2k+1 states by transitions, as described above.

• For each transition �(s, x) = (s0, x0, pos) of M , the machine M 0 must:(i) recognize x, (ii) override x0, (iii) move according to pos and go tostate s0. Thus:

– (i) is done by the procedure described at the above point.

– for (ii), we use k states to go back (k cells) at the beginning ofenc⌃0(x) and write enc⌃0(x0), cell by cell. Finally, we connect thestate corresponding to enc⌃0(x) from the tree, to the first of theabove-described k states.

– for (iii) if pos = L/R, we use another k states to go either left orright. If pos = H, we need not use these states. Finally, we needto make a transition to the root of the state tree correspondingto s0.

For each transition �(s, x) = (s0, x0, pos) of M , M 0 performs k transitionsfor reading the encoded symbol x, k transitions for writing x0 and possiblyk transitions for moving the tape head. Thus, for all w 2 ⌃0, the numberof transitions performed by M 0 is at most 3k ⇤ T (|w|). Hence, the runningtime of M 0 is O(k) ⇤ T (n). ⇤

The proof of Proposition 0.3.2.1 shows that any Turing Machine using anarbitrary alphabet ⌃ can be transformed in one using the binary alphabet.The overhead of this transformation is logarithmic: O(dlog |⌃|e). Thus, ifthe original Turing Machine runs in some polynomial time T (n), then thetransformed TM will run in O(dlog |⌃|e) ⇤ T (n) time which is bound by apolynomial. Similarly, if the original TM is running in supra-polynomialtime, the transformed TM will also run in supra-polynomial time.

In what follows, we shall assume all Turing Machines are using the binaryalphabet ⌃b = {0, 1,#, >}.

0.3.3 Complexity classes

In the previous section, we have stated that, in complexity theory, we shallmake no distinction between polynomial running times with di↵erent asymp-totic growths (e.g. between n2 and n3). With this in mind, we construct aclassification of problems. First, we say that: f is decidable in time T (n)i↵ there exists a Turing machine M which decides f , and whose runningtime is T . We interchangeably use the terms decidable and solvable, since,

32

in this chapter, there is no ambiguity on what “solvability” means, and itcannot be mistaken by acceptability. All considered problems in this sectionare members of R. The following definition characterizes problems with aspecific running time.

DTIME(T (n)) = {f : N ! {0, 1} | f is decidable in time O(T (n))}

Note that, unlike asymptotic notations, DTIME(T (n)) is a class of prob-lems, not of running times. Also, note that our characterization does notprovide a strict upper bound. Hence, e.g. DTIME(n) ✓ DTIME(n2). Inwords: a problem which is decidable in liniar time, is also decidable inquadratic time. Next, we introduce the class:

PTIME =[

d2NDTIME(nd)

PTIME is often abbreviated P. It is the class of all problems which aredecidable in polynomial time. Also, note that if some problem f is decidablein log(n) time, then f 2 DTIME(log(n)) ✓ DTIME(n) ✓ P. Hence, evenproblems which are solvable in sub-liniar time belong in the class P.

Further on, we introduce the class:

EXPTIME =[

d2NDTIME(2n

d)

which contains all the problems which are decidable in exponential time.Naturally, we have:

P ✓ EXPTIME ✓ R

There are two interesting questions which can be raised, at this point:

1. Is the inclusion P ✓ EXPTIME strict?

2. Are all problems in EXPTIME \ P8, “equals”, in terms of di�culty?

In the following section, we shall focus on the latter question:

Nondeterminism and Nondeterministic Turing Machines

We recall the problem SAT, which takes as input a boolean formula inConjunctive Normal Form (CNF). More precisely:

= C1 ^ C2 ^ . . . ^ Cn

8Later in this chapter, we shall see that EXPTIME \ P is a set whose members arecurrently unknown.


where, for each i : 1 i n we have

Ci = Li1 _ Li2 _ . . . _ Limi

and, for each j : 1 j mi we have

Lij = x or Lij = ¬x

and finally, x is a variable.Recall that SAT can be solved in exponential time, hence SAT 2 EXPTIME.

The major source of complexity consists in generating all possible interpre-tations on which a verification is subsequently done. In order to answerquestion 2. from the previous section, suppose that SAT would be solvablein polynomial time. Could we find problems (possibly related to SAT) whichare still solvable in exponential time under our assumption? The answer isyes: Let � be the formula:

� = 8x18x2 . . . 8xk

where is a formula in CNF containing variables x1, . . . , xk, and k 2 Nis arbitrarly fixed. Checking if � is satisfiable is the problem 8SAT. Analgorithm for 8SAT must build all combinations of 0/1 values for each xiwith i : 1 i k and for each one, must solve an instance of the SATproblem. In total, we have 2k combinations, and since k is part of the input,the algorithm runs in exponential time, provided that we have an algorithmfor SAT which runs in polynomial time.

In order to study the di�culty of problems, in Complexity Theory, wegeneralise the above-presented approach, in order to determine degrees ofhardness. In order to do so, we need a general version of our assumption:“SAT is solvable in polynomial time”. In other words, we need a theoreticaltool to make exponential search (seem) polynomial. This tool will have norelation to reality, and no implications for real problem solving. It shouldnot be understood as a technique for deciding hard problems. It’s merepurpose is theoretical: it allows us to abstract one source of complexity(that of SAT in our example) in order to explore others (e.g. 8SAT). Thetool that we mentioned is the Nondeterministic Turing Machine:

Definition 0.3.3.1 (Non-deterministic TM) A non-deterministic Tur-ing machine (NTM short) is a tuple M = (K,F,⌃, �, s0) over alphabet ⌃with K,⌃ and s0 defined as before, F = {syes, sno} and � ✓ K ⇥ ⌃ ⇥K ⇥⌃⇥ {L,H,R} is a transition relation.

34

A NTM terminates i↵ it reaches a final state, hence, a state in F . ANTM M decides a function f : N ! {0, 1} i↵ f(nw) = 0 =) M(w)reaches state sno on all possible sequences of transitions and f(nw) =1 =) M(w) reaches state syes on at least one sequence of transitions.

We say the running time of a NTM M is T i↵, for all w 2 ⌃, allsequences of transitions of M(w) contain at most T (|w|) steps.

We start with a few technical observations. First note that the NTMis specifically tailored for decision problems. It has only two final states,which correspond to yes/no answers. Also, the machine does not producean output, and the usage of the tape is merely for internal computations.In essence, these “design choices” for the NTM are purely for convenience,and alternatives are possible.

Whereas the conventional Turing Machine assigned, for each combina-tion of state and symbol, a unique next-state, overriding symbol and headmovement, a nondeterministic machine assigns a collection of such elements.

The current configuration of a conventional Turing Machine was charac-terized by the current state, by the current contents of the tape, and by thehead position. A configuration of the nondeterministic machine correspondsto a set of conventional TM configurations. The intuition is that the NTMcan simultaneously process a set of conventional configurations, in one sin-gle step. While the execution of a Turing Machine can be represented as asequence, that of the NTM can be represented as a tree. A path in the treecorresponds to one sequence of transitions which the NTM performs.

Now, notice the conditions under which a NTM decides a function: if atleast one sequence of transitions leads to syes, we can interpret the answerof the NTM as yes. Conversely, if all sequences of transitions lead to sno,then the machine returns no.

Finally, when accounting for the running time of a NTM, we do notcount all performed transitions (as it would seem reasonable), but only thelength of the longest transition sequence performed by the machine. Theintuition is that all members of the current configuration are processed inparralel, during a single step.

We illustrate the NTM in the following example:

Example 0.3.3.1 (Nondeterministic Turing Machine) We build the NTMMSAT which solves the SAT problem discussed previously. First, we assumethe existence of Mchk(I, ) which takes an interpretation and a formula, bothencoded as a unique string, and checks if I |= . Mchk is a conventionalTM, thus upon termination it leaves 0 or 1 on the tape.


Step 1: MSAT computes the number of variables from (henceforth referredto as n), and prepends the encoding of with the encoding of n inunary (as a sequence of 1’s). This step takes O(n) transitions.

Step 2: During the former step, MSAT has created a context for generating in-terpretations. In this step, MSAT goes over each cell from the encodingof n, and non-deterministically places 0 or 1 in that cell. Thus, af-ter 1 such transition, there are 2 possible conventional configurations.In the first, bit 0 is placed on the first cell of the encoding of n. Inthe second, bit 1 is placed in the same position. After i transitions,we have 2i possible configurations. At the end of this step, we have2n possible configurations, and in each one, we have a binary word oflength n at the beginning of the tape, which corresponds to one possibleinterpretation. All sequences of transitions have the same length, thus,the execution of this part of MSAT takes O(n).

Step 3: At the end of each sequence illustrated above, we run Mchk(I, ), whereI and are already conveniently on the tape. This step takes O(n⇤m)where m is maximal number of literals in .

If we add up all running times of the three steps, we obtain O(n ⇤m).

An important issue is whether the NTM has more expressive power thanthe conventional TM:

Proposition 0.3.3.1 Every function which is decidable by an NTM in poly-nomial running time, is also decidable by a TM which runs in exponentialtime.

The proof is left as exercise. Intuitively, we can simulate a NTM by doinga backtracking procedure, with a classic TM. The propositions shows thatthe NTM only o↵ers a gain in speed and not expressive power. It solvesprecisely the same problems which the conventional TM solves.

Convention for describing NTM in pseudocode. In the previouschapters, we often resorted to traditional pseudocode in order to describealgorithms – that is, Turing Machines. It is occasionaly useful to be ableto do the same thing for NTMs. With this in mind, we introduce somenotational conventions. The instruction:

v = choice(A)

where v is a variable and A is a set of values, behaves as follows:

36

• the current (non-deterministic) configuration of the NTM shall contain|A| conventional configuration.

• each conventional configuration corresponds to a distinct value a 2 A,and it should be interpreted that v = a, in that particular configura-tion.

• the running time of the instruction is O(1).

We also note that, it is not possible to achieve some form of “communica-tion” between conventional configurations. Thus, it is intuitive to think thatthe processing (execution of a transition) of a conventional configuration isdone independently of all other conventional configurations.

We add two aditional instructions: success and fail. They correspondto a transitions into states syes and sno, respectively.

We illustrate NTM pseudocode, by re-writing the SAT algorithm de-scribed above. We adopt the same representational conventions from thefirst Chapter, and also re-use the procedure CHECK.

Example 0.3.3.2 (Pseudocode) .SolveSAT('):

Let n be the number of variables in '.

Let I be a vector with n components which are initialised with 0

for i = 0, n� 1:

I[i] = choice({0, 1})

if CHECK(',I) = 0

fail

else success

As illustrated in the previous example, the NTM has the power of tam-ing down the complexity which results from the search of an exponentialnumber of candidates (in our example, interpretations). Note that, in theNTM, the main source of complexity is given by the verification procedureof Mchk(I, ).

By using the NTM, we have managed to find the source of complexityof SAT, which is the exponential search over possible interpretations. Wehave seen that there are other types of exponential explosion. Thus, we are


in the position for refining our classification, by introducing new complexityclasses:

NTIME(T (n)) = {f : N ! {0, 1} | f is decidable by a NTM in time O(T (n))}

and

NPTIME =[

d2NNTIME(nd)

We usually abbreviate the class NPTIME by NP. Note that SAT 2 NP,however it seems unlikely that 8SAT 2 NP. We shall discuss this issue inthe next section. Le us relate NP with our other classes. First, note thatNP ✓ EXPTIME. This result is essentially given by Proposition 0.3.3.1:every problem solved by a NTM can be solved in exponential time by a TM.Also, we trivially have: P ✓ NP. The concise argument is that the NTM isa generalization of the TM, “minus” some technical details. Thus:

P ✓ NP ✓ EXPTIME ✓ R

The fundamental property of problems from NP is that “solution can-didates can be verified in polynomial time”. An analogy with solving cross-words is possible. Generating all possible solutions (by “lexicographically”generating all posible words) is obviously exponential. But since verifiyingwhether a solution is correct can be done in polynomial time w.r.t. the size ofthe crossword, the problem in in NP. We shall soon see that all algorithmswhich solve hard problems from NP can be split up into an exponential“generating” procedure, and a polynomial “verification procedure”.

0.3.4 Hard and complete problems for a class

Recall that proving f 62 R for a given problem f was done using contrapo-sition. Essentially, the proof relied on finding a Turing reduction f⇤ T fto a problem f⇤ for which f⇤ 62 R is known.

First, we note that R could be easily replaced with any complexity classX. Thus, a more general proof scheme could be described as:

f⇤ 62 X, f X f⇤ =) f 62 X

where f⇤ and X are given in advance. We observe that we have replaced T

by X , in order to highlight that the reduction X depends on the choice

Turcu George

38

of X. We shall later see that we cannot allways use Turing Reductions, andthere are X’s for which more restrictions on the reduction must be in place.

Returning to the proof scheme, we make a second note: to prove f 62 R,we must first find some f⇤ 62 R. But if this very fact is shown by the sametechnique, then we need another problem f⇤⇤ 62 R and so on. This situationis similar to: “Which came first, hens or eggs?”.

For the case of R, this issue was settled by the proof fh 62 R usingdiagonalization. This introduced an initial undecidable problem, and theaforementioned technique could be employed.

Now, let us turn our attention to X = NP. First, let us examine theproperties which the reduction should have, in order to make our techniquework in this particular case. let f⇤ 62 NP. We need to show that f 62 NP.The first part consists of assuming f 2 NP. Next, we a reduction T : If⇤ !If where If⇤ and If are the inputs of the problem from the subscript.9 Werecall that the reduction needs to satisfy:

8i 2 If⇤ : f⇤(i) = 1 () f(T (i)) = 1

In words: “we can solve all instances i of If⇤ by solving instances T (i)of If”. This is done as follows:

a. receive i 2 If⇤

b. compute T (i) 2 If

c. run the NTM Mf (T (i)) (Mf must exist since f 2 NP) and return it’sanswer.

After a careful look, we observe that the conclusion f⇤ 2 NP (whichis our objective, in order to complete the contrapositive argument) is notimmediate. If, for instance, the computation of T (i) takes exponential time,then the proof does not work: the NTM which performs a. b. c. runs in(non-deterministic) exponential time.

Thus, the restriction that T is decidable is insu�cient. We further needthat T is computable in polynomial time. We write f⇤ p f i↵ thereexists a polynomial-time transformation which allows solving f⇤ via f , asillustrated by the scheme above.

We now turn our attention to the “hen and eggs” issue. We need aninitial problem, which is known not to be in NP. Unfortunately, such a

9In order to make explicit the direction of the transformation, we ignore the fact thatboth If⇤ and If are (subsets of) naturals.


problem is not known, although there have been major (unsuccessful) e↵ortsin trying to find one. The same holds for the class P. Hence, the issue:

P ( NP

is currently open, and it is generally believed that it is true, but all proofattempts have failed. The good news is that our e↵ort in classifying problemsis not fruitless. Transformations:

f⇤ p f f⇤ T f

establish a relation between problems, which is denoted as hardness. Nomatter the reduction type (polynomial or Turing), f is at least as hard asf⇤. With the machine Mf (together with computing a transformation) wecan solve all instances of f⇤. It may be possible that T is bijective: henceeach input of f⇤ is uniquely mapped to an input of f and vice-versa. Then,f and f⇤ are equally hard. However, this is not generally the case, hence theterm “at least” as hard.

We can naturally extend hardness to complexity classes:

Definition 0.3.4.1 A problem f is called NP-hard i↵ for all f 0 2 NP,f 0 p f .

Thus, a problem is hard w.r.t. a class, i↵ it is at least as hard as anyproblem in the class at hand. Note that hardness can be defined w.r.t. anycomplexity class and not just NP, provided that the appropriate type oftransformation is employed.

Definition 0.3.4.2 A problem f is called NP-complete i↵ it is NP-hardand f 2 NP.

Informally, NP-complete problems are the hardest problems in NP. Inthe more general case, complete problems w.r.t. a class are the hardest ofthat class.

It is likely that if f is NP-complete, then f 62 P , however, this is not aproven fact.

Thus, instead of trying to disprove membership of a class (f 62 P ),in complexity theory, we prove completeness for the immediate upper (orgreater) class (f is NP-complete).

The intuition is that class membership provides an upper bound, whilehardness — a lower bound for the di�culty of a problem. We illustrate thisby an abstract example. Recall:

P ✓ NP ✓ EXPTIME

Turcu George

Turcu George

40

Let f be a problem. Suppose we find an algorithm for f which runs inexponential time, however, we cannot find one which runs in polynomialtime on a NTM. At this point, we have f 2 EXPTIME. Suppose we showf is P-hard, thus, f can be used to solve any problem in P. We now knowthat f can be solved exponentially and it is unlikely that f can be solvedin sub-polynomial (e.g. logarithmic) time. Thus, the likely variants are: fmay be solved in polynomial time (i) by a conventional TM f 2 P or (ii) bya NTM f 2 NP, and (iii) in exponential time, again by a conventional TMf 2 EXPTIME. In the best case f is polynomially solvable. In the worst— it is exponentially solvable.

Suppose now we also find that f is NP-hard. We cannot rule out f 2 Pby a proof, but Complexity Theory predicts that such a membership is notlikely. Hence, the feasible variants remain (ii) and (iii).

Finally, if we manage to improve our exponential algorithm for f andturn it into a non-deterministic polynomial algorithm, then f 2 NP and,hence, it is NP-complete. Case (iii) remains of course true, but it does notcarry useful information. At this point, we have an exact characterisationof the di�culty of f .

Proving NP-completeness For a problem f to be NP-complete, it mustsatisfy two conditions. The first: f 2 NP is shown by finding a NTMwhich decides f in polynomial time. For the second part (f is NP-hard),we can employ precisely the “reduction-finding” technique illustrated at thebeginning of this section.

Proposition 0.3.4.1 A problem f is NP-hard i↵ there exists a problem gwhich is NP-hard, such that g p f .

Proof: Suppose g p f and g is NP-hard. Hence, for all h 2 NP, h p g.By transitivity, we also have that h p f . It follows that f is NP-hard. ⇤

In the former proof we have made use of the transitivity of p, withoutshowing it. We now state several properties of o including transitivity, andleave the proofs as exercises.

Proposition 0.3.4.2 p is reflexive and transitive.

Proposition 0.3.4.3 The set of NP-hard problems is closed under p.

Proposition 0.3.4.4 The set of NP-complete problems together with p isan equivalence class.

Turcu George

Turcu George


Proposition 0.3.4.5 Assume f is NP-complete. If f 2 P, then P = NP.

The former proposition, whose proof follows immediately from the underly-ing definitions, makes the case for the common belief that P 6= NP. If somee�cient algorithm can be found for some NP-complete problem, then allproblems in NP can be solved in polynomial time.

The P = NP issue can also be given another intuitive interpretation:“The verification of a solution candidate is as di�cult as generating it” or,alternatively: “Verifying a given proof P for A, is as di�cult as finding aproof for P”.

Finally, to better understand the implications of P = NP, consider sev-eral facts which would be arguably true, in the case the former equalityholds:

• We can provide a solution to the astronauts’ problem (see first chapter)

• Partial program correctness can be solved e�ciently. Technique suchas model checking can be applied to a wide range of applications (in-cluding operating system kernels). Bugs are almost removed. Win-dows bluescreens are no longer happening.

• Generation of exponentially many training sets would make tasks suchas voice recognition, computer vision, natural language processing —computationally easy.

• Mathematical proofs (of, say 100 pages) can be generated e�ciently.Computers can be used to find proofs for some open problems.

• We can exponential search to find passwords, or to break encryptionkeys in polynomial time. Internet privacy is no longer possible usingencryption (e.g. using SSH). Internet commerce and banking is nolonger possible. Safe communication is no longer possible (at all lev-els). Any computer-controlled facility (public, military, etc.), which isconnected to the Internet has considerable potential of being compro-mised.

Remark 0.3.4.1 (Practical applications of reductions) As illustratedbefore, reductions of the type p are a theoretical tool which is useful for prov-ing NP-hardness. Reductions also have practical applications. For instance,most NP-complete problems are solved by employing SAT solvers, which,as discussed in the former chapters, may be quite fast in the general case.Thus, a specific problem instance is cast (via an appropriate transformation)

42

into a formula ', such that ' is satisfiable i↵ the answer to the instance isyes.

SAT. The first NP-complete problem. We observe that the “hen andeggs” issue still holds in our scenario. To apply our technique, we needan initial NP-hard problem in the first place. This problem is provided byCook’s Theorem, which proves that SAT is NP-complete. The technique forthe proof relies on building, for each NTM M (and hence, for each problemin NP), a formula 'M such that it is satisfiable i↵ there exists a sequence inthe computation tree leading to success.

Bibliography

[1] A.J. Hildebrand. Asymptotic methods in analysis -http://www.math.uiuc.edu/h̃ildebr/595ama/. Math595AMA, 2009.

[2] Christos M. Papadimitriou. Computational complexity. Addison-Wesley,Reading, Massachusetts, 1994.

43

Suport de Curs-AA

Documents

Transcript of Suport de Curs-AA