Week 10Complexity of Algorithms1 Hard Computational Problems Some computational problems are hard...

week 10 Complexity of Algorithms 1

Hard Computational Problems

Some computational problems are hardDespite a numerous attempts we do not know any efficient algorithms for these problemsWe are also far away from the proof that these problems are indeed hard to solve, in other wordsNP=P or NPP, this is a question …


Decision/Optimisation Problems

A decision problem (DP) is a computational problem for which the intended output is either yes or noIn an optimisation problem (OP) we rather try to maximise or minimise some valueAn OP can be turned into a DP if we add a parameter k, and then ask whether the optimal value in OP is at most or at least kNote that if a DP is hard, then its related its optimisation version must be hard too


Decision/Optimisation Problems

Example:Optimisation problem - Given graph G with integer weights on its edges. What is the weight of a minimum spanning tree (MST) in G?Decision problem – Given graph G with integer weights on its edges, and an integer k. Does G have a minimum spanning tree of weight at most k?


Problems and Languages

We say that the algorithm A accepts an input string x if A outputs yes on input xA decision problem can be viewed as a set L of (binary) strings – the strings that should be accepted by an algorithm that correctly solves the problemWe often refer to L as a languageWe say that an algorithm A accepts a language L if A outputs yes for each x in L and outputs no otherwise


The Complexity Class P

The complexity class P is the set of all decision problems (or languages) L that can be solved in worst-case polynomial timeThat is, there is an algorithm A that if x L, then on input x, A outputs yes in time p(n), where n is the size (length) of x and p(n) is a polynomial


The Complement of a Language

The complement of a language L consists of all strings that are not in LIf we have a p(n) time algorithm A that accepts L (i.e., L is in P) we can construct a p(n) time algorithm B (based on A) that accepts the complement of L, i.e.,

Run algorithm A on input string x for p(n) stepsIf A outputs yes, then B outputs noIf A outputs no or give no output, then B outputs yes

If L is in P, the complement of L is in P too!


The Complexity Class NP

An algorithm that chooses (by a really good guess!) some number of non-deterministic bits during its execution is called a non-deterministic algorithmWe say that an algorithm A non-deterministically accepts a string x if there exists a choice of non-deterministic bits that leads to the ultimate answer yesThe complexity class NP is the set of decision problems (or languages) L that can be non-deterministically accepted in polynomial time Obviously P NP


The Complement of L in NP

Note that the definition of class NP does not address the running time of rejection (which might be very long)And indeed even knowing that we can choose an appropriate number of non-deterministic bits for all strings in L in NP we cannot assure that such a choice is feasible for the complement of LIn fact there is a class co-NP that consists of all languages whose complements are in NP Many researchers believe that co-NP NP


The P = NP Question

Computer scientists do not know for certain whether P = NP or notWe also do not know whether P = NP co-NPHowever there is a common believe that P is different then both NP and co-NP, as well as they intersection


Hamiltonian Cycle is NP

Hamiltonian-Cycle is the problem that takes a graph G as an input and asks whether there is a simple (Hamiltonian) cycle in G that visits every vertex of G exactly onceThe non-deterministic algorithm chooses a cycle (represented by a sequence of non-deterministic bits) and then it checks deterministically whether this cycle is indeed a Hamiltonian cycle in G


Boolean Circuit

A Boolean circuit is a directed graph where each node, called a logic gate corresponds to a simple Boolean function AND, OR, or NOTThe incoming edges for a logic gate correspond to inputs for its Boolean function and the outgoing edges correspond to the outputs


Boolean Circuit (example)


Circuit-SAT is in NP

Circuit-Sat is the problem that takes an input a Boolean circuit with a single output node, and asks whether there is an assignment of values to the circuit’s inputs so that its output value is 1The non-deterministic algorithm chooses an assignment of input bits (represented by a sequence of non-deterministic bits) and then it checks deterministically whether this input generates output 1


Vertex Cover

Given a graph G=(V,E), a vertex cover for G is a subset CV, s.t.,for every edge (v,w) in E, vC or wC

The optimisation problem is to find as small a vertex cover as possibleVertex-Cover is the decision problem that takes a graph G and an integer k as input, and asks whether there is a vertex cover for G containing at most k vertices


Vertex-Cover is in NP

Suppose we are given an integer k and a graph GThe non-deterministic algorithm chooses a subset of vertices C V, s.t., ¦C¦ k, (represented by a sequence of non-deterministic bits) and then it checks deterministically whether this subset C is an appropriate vertex cover


Polynomial-Time Reducibility

We say that a language L, defining some decision problem, is polynomial-time reducible to a language M, if

there is a function f computable in polynomial time, thattakes an input x to L, and transforms it to an input f(x) of M, s.t.,x L if and only if f(x) M

We use notation L poly M to signify that language L is polynomial-time reducible to language M


NP-hardness

We say that a language M, defining some decision problem, is NP-hard if every other language L in NP is polynomial-time reducible to M, i.e.,M is NP-hard, if for every L NP, L poly M If a language M is NP-hard and it belongs to NP itself, then M is NP-completeNP-complete problem is, in a very formal sense, one of the hardest problems in NP, as far as polynomial-time reducibility is concerned


The Cook-Levin Theorem

The Cook-Levin TheoremCircuit-Sat is NP-complete

Proof [sketch]: A computation steps of any (reasonable) algorithm can be simulated by layers in appropriately constructed (in polynomial time and size) Boolean circuit


The Cook-Levin Theorem


Other NP-complete Problems

We have just noted that there is at least one NP-complete problemUsing polynomial-time reducibility we can show existence of other NP-complete problems according toLemma:

If L1 poly L2 and L2 poly L3 then L1 poly L3


Types of reduction

Let M be known NP-complete problem. The types of reductions are:

By restriction: noting that known NP-complete problem is a special case of our problem LLocal replacement: dividing instances of M and L into basic units, and then showing how each basic unit of M can be locally converted into a basic unit of LComponent design: building components for an instance of L that will enforce important structural functions for instances of M


Important NP-complete Problems


Conjunctive Normal Form

A Boolean formula is in conjunctive normal form (CNF) if it is formed as a collection of clauses combined using operator AND (·), where each clause is formed by literals (variables or their negations) combined using operator OR (+), e.g.,


CNF-SAT & 3SAT

Problem CNF-SAT takes a Boolean formula in CNF form as input and asks if there is an assignment of Boolean values to its variables so that the formula evaluates to 1 (i.e., formula is satisfiable)3SAT is CNF-SAT in which each clause has exactly 3 literalsFact: CNF-SAT and 3-SAT are NP-complete


Vertex-Cover is NP-complete

We can show that Vertex-Cover is NP-hard by reducing 3SAT problem to it in polynomial timeThis reduction is an example of a reduction from a logic problem to a graph problemIt also illustrates an application of the component design proof technique


Approximation Schemes

One way of dealing with NP-completeness for optimisation problems is to use an approximation algorithmThe goal of an approximation algorithm is to come as close to the optimum value as possible Such an algorithm typically runs much faster than an algorithm that strives for exact solution


Approximation Schemes

Let c(S) be the value of a solution S delivered by an algorithm A to an optimisation problem P and OPT will be the optimal solution for P We say that A is a -approximation algorithm for a minimisation problem P if

c(S) ·OPT

And A is is a -approximation algorithm for a maximisation problem P if

c(S) ·OPT


Polynomial-Time Approximation Scheme (PTAS)

There are some problems for which we can construct -approximation algorithms that run in polynomial time with =1+, for any fixed value > 0The running time of such collection of algorithms depends both on n, the size of an input and also on a fixed value We refer to such collection of algorithms as a polynomial-time approximation scheme, or PTASIf the running time is polynomial in both n and 1/ we have a fully polynomial-time approximation scheme

Week 10Complexity of Algorithms1 Hard Computational Problems Some computational problems are hard...

Documents

Transcript of Week 10Complexity of Algorithms1 Hard Computational Problems Some computational problems are hard...