The classes NP , and coNP. More Problems in NP. Poly...

Computational Models Lecture 12

The classes NP , and coNP.

More Problems in NP.

Poly-Time Reductions

NP completeness

SAT is NP Complete

Sipser, Chapter 7, Sections 7.3, 7.4, 7.5

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 1

Non-Deterministic Time (reminder)

Let N be a non-deterministic TM, and let

f : N −→ N

We say that N runs in time f(n) if

For every input x of length n,

the maximum number of steps that N uses,

on any branch of its computation tree on x,

is at most f(n).


NTime Classes Definition (reminder)

Letf : N −→ N

be a function.

Definition:

NTIME(f(n)) = {L | L is a language, decidedby an O(f(n))-time NTM}


The Class NP (reminder)

Definition: NP is the set of languages decidable inpolynomial time on non-deterministic TMs.

NP =⋃

c≥0

NTIME(nc)

The class NP is

Invariant for all TMs with any number of tapes.

Insensitive to choice of reasonable non-deterministiccomputational model.

Rougly corresponds to problems whose positivesolutions cannot be efficiently generated(⇒ intractable), but can be efficiently checked.


The Class NP

NP is important because it includes many problems ofpractical interest, e.g

Hamiltonian path

Travelling salesman (salesperson, that is)

Scheduling (operations research)

Placement and routing (VLSI design)

Composites (factoring/cryptography)

...


Verifiability (reminder)

A verifier for a language A is an algorithm V where

A = {w | V accepts 〈w, c〉 for some string c}

The verifier uses the additional information c to verifyw ∈ A.

We measure verifier run time by length of w.

The string c is called a certificate (or proof) for w if Vaccepts 〈w, c〉.

A polynomial verifier runs in polynomial time in |w| (so|c| ≤ |w|O(1)).

A language A is polynomially verifiable if it has apolynomial verifier.


NP and Verifiability (reminder)

Theorem: A language is in NP if and only if it has apolynomial time verifier.

Proof – Intuition:

NTM simulates verifier by guessing the certificate.

Verifier simulates NTM by using accepting branch ascertificate.


NP

Claim: If A has a poly-time verifier, then it is decided bysome polynomial-time NTM.

Let V be poly-time verifier for A.

single-tape TM

runs in time nk

N : on input w of length n

Nondeterministically select string c of length nk.

Run V on 〈w, c〉

If V accepts, accept; otherwise reject.


NP

Claim: If A is decided by a polynomial-time NTM N ,running in time nk, then A has a poly-time verifier.Construct polynomial-time verifier V as follows.

V: on input w of length n, and on a string c of length nk

Simulate N on input w, treating each symbol of c as adescription of the non-deterministic choice in each stepof N .

If this branch accepts, accept, otherwise reject.


Examples: Clique

A clique in an undirected graph is a subgraph where everytwo nodes are connected by an edge.

A k-clique is a clique of size k.What is the largest k-clique in the figure?(ignore the colors of edges)


Examples: Clique

Define the language

CLIQUE = {〈G, k〉|G is an undirected graphwith a k-clique}


Examples: Clique

Theorem:CLIQUE ∈ NP

The clique is the certificate.

Here is a verifier V: on input (〈G, k〉, c)

if c is not a k-clique, reject

if G does not contain all vertices of c, reject

accept


Example: SUBSET-SUM

An instance of the problem

A collection of numbers x1, . . . , xk

Target number t

Question: does some subcollection add up to t?

SUBSET-SUM = {〈S, t〉 | S = {x1, . . . , xk}

∃ {y1, . . . , yℓ} ⊆ {x1, . . . , xk},∑

yj

= t}

Collections are sets: repetitions not allowed.


Example: SUBSET-SUM

We have

({4, 11, 16, 21, 27} , 25) ∈ SUBSET-SUM

because 4 + 21 = 25.

({4, 11, 16, 21, 27} , 26) /∈ SUBSET-SUM

(why?)


Example: SUBSET-SUM

Theorem:SUBSET-SUM ∈ NP

The subset is the certificate.

Here is a verifier:V: on input (〈S, t〉, c)

test whether c is a collection of numbers summing to t.

test whether c is a subset of S

if either fail, reject, otherwise accept.


Complementary Problems

CLIQUE and SUBSET-SUM seem not to be members of NP.

It is harder to efficiently verify that something does not existthan to efficiently verify that something does exist..

Definition: The class coNP:L ∈ coNP if L ∈ NP.


P vs. NP

NP

P P=NP

The question P = NP? is one of the great unsolvedmysteries in contemporary mathematics.

most computer scientists believe the two classes arenot equal

most bogus proofs show them equal (why?)Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 17

P=NP?

If P=NP, then the world would be a profoundly differentplace than we usually assume it to be. There would beno special value in “creative leaps", no fundamental gapbetween solving a problem and recognizing the solutiononce it’s found. Everyone who could appreciate asymphony would be Mozart, everyone who could followa step-by-step argument would be Gauss..." ScottAaronson, MIT.

The main argument in favor of P6=NP is the total lack offundamental progress in the area of exhaustive search.THis is, in my opinion, a very weak argument. Thespace of algorithms is very large and we are only at thebeginning of its exploration. Moshe Vardi, Rice University.


Observations

If P differs from NP , then the distinction between P andNP − P is meaningful and important.

languages in P tractable

languages in NP − P intractable

Until we can prove that P 6= NP , there is no hope of provingthat a specific language lies in NP − P .Nevertheless, we can prove statements of the form “IfP 6= NP then A ∈ NP − P .”


NP Completeness

P

NP

NP-complete

The class of NP-complete languages are

“hardest” languages in NP

“least likely” to be in P

If any NP-complete A ∈ P , then NP = P .


Cook–Levin (1971-1973)

Theorem: There is a language S ∈ NP such that:S ∈ P if and only if P = NP .

This theorem establishes the class of NP-completelanguages. Such languages “carry on their backs” theburden of all of NP .


Poly-Time Computable Functions

Definition: A function

f : Σ∗ −→ Σ∗

is polynomial-time computable if there is a poly-timedeterministic TM that

starts with input w, and

halts with f(w) on tape.


Poly-Time Reducibility

Definition: We say that a language A is polynomial timemapping reducible to B, written

A ≤P B,

if there is a poly-time computable function

f : Σ∗ −→ Σ∗

such that, for every w,

w ∈ A ⇐⇒ f(w) ∈ B.

The function f is called a polynomial-time reduction from Ato B.



Convert questions about membership in A to membershipin B, and do it efficiently.



Theorem: If A ≤P B and B ∈ P then A ∈ P .

Proof: Let

f the reduction from A to B, computed by TM Mf .

On input x of length n, Mf takes at most c1na1 steps.

M be the poly-time decider for B.

On input y of length m, M takes at most c2ma2 steps.



Define N : on input x

1. compute f(x)

2. run M on input f(x) and output whatever M outputs.

Analysis:

On input x of length n, computing y = f(x) takes atmost c1n

a1 steps.

On input y of length m = c1na1, M takes at most

c2ma2 = c2(c1n

a1)a2 = (c2ca2

1 )na1·a2 steps.

Summing both stages, we got a polynomial in n.

Correctness is clear, so A ∈ P . ♣


Satisfiability

A Boolean variable assumes valuestrue (written 1), and false (written 0).

Boolean operations:and: ∧or: ∨not: ¬

Examples:

0 ∧ 1 = 0

0 ∨ 1 = 1

0 = 1


Satisfiability

A Boolean formula is an expression involving Booleanvariables and operations.

φ = (x ∧ y) ∨ (x ∧ z)

Definition: A formula is satisfiable if some assignment of 0sand 1s to the variables makes the formula evaluate to 1.


Satisfiability

φ = (x ∧ y) ∨ (x ∧ z)

is satisfiable by

x = 0

y = 1

z = 0

This assignment satisfies φ.


Satisfiability

Define

SAT = {〈φ〉 | φ is satisfiable Boolean formula}


Satisfiability

It is useful to consider a special version:

A literal is a variable or negated variable: x or x.

A clause is several literals joined by ∨s: (x1 ∨ x2 ∨ x3)

A Boolean formula is in conjunctive normal form (CNF)if it consists of clauses, connected with ∧s.

For example

(x1 ∨ x2 ∨ x3 ∨ x4) ∧ (x3 ∨ x5 ∨ x6) ∧ (x3 ∨ x6)


Satisfiability

Definition: A Boolean formula is in 3CNF form if it is a CNFformula, and all clauses have three literals.

(x1 ∨ x2 ∨ x3) ∧ (x3 ∨ x5 ∨ x6) ∧ (x3 ∨ x6 ∨ x4)

Define

3SAT = {〈φ〉 | φ is satisfiable 3CNF formula}

Clearly, if φ is a satisfiable 3CNF formula, then for anysatisfying assignment of φ, every clause must contain atleast one literal assigned 1.


Reductions

Claim: There is a poly time reduction from 3SAT toCLIQUE. In other words,

3SAT ≤P CLIQUE .

We’ll construct a poly time reduction f that maps 3CNFformulae φ to graphs and numbers, 〈G, k〉.

The function f will have the property that φ is satisfiable ifand only if G has a clique of size k.


Examples: Clique

Reminder: A clique in a graph is a subgraph where everytwo nodes are connected by an edge.A k-clique is a clique of size k. For example, the graphabove has a 5-clique.


3SAT ≤P CLIQUE

Let φ be a 3CNF formula with k clauses.

(x1 ∨ x2 ∨ x3) ∧ (x3 ∨ x5 ∨ x6) ∧ (x3 ∨ x6 ∨ x4)

We define a graph G as follows:

nodes in G are organized into triples t1, . . . , tk.

each triple corresponds to a clause of φ

each node in a triple corresponds to a literal incorresponding clause.


3SAT ≤P CLIQUE


3SAT vs. CLIQUE

Now add edges between all vertex pairs, except

within same triple

between contradictory literals

Note: No edges along “circle”. These are only for illustration.Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 37

3SAT ≤P CLIQUE

Claim: If φ is satisfiable, G has a k-clique.

Suppose φ is satisfiable.

at least one literal is true in every clause

in every tuple, select one true literal

they can be joined by edges

yielding a k-clique


3SAT ≤P CLIQUE

Claim: If φ is satisfiable, G has a k-clique.


3SAT ≤P CLIQUE

Claim: If G has a k-clique, φ is satisfiable.

No two of the clique’s nodes are in the same triple.

Have k vertexes and k clauses, so

each triple has exactly one clique node.

Assign 1 to each node in clique

no contradictions.


3SAT ≤P CLIQUE

We’ve constructed a poly time computable function f .

We saw that the function f has the property that φ ∈3SAT if and only if f(φ) ∈ CLIQUE.

Therefore f is a polynomial time reduction from 3SAT toCLIQUE, so 3SAT ≤P CLIQUE. ♣


Independent Set

An independent in a graph is a set of vertexes, no two ofwhich are linked by an edge.

The independent set problem asks whether there exists anindependent set of size k.


Independent Set

Define

INDEPENDENT-SET ={〈G, k〉|G contains an independent set of size k}

Claim: INDEPENDENT-SET is polynomial time reducible toCLIQUE,

INDEPENDENT-SET ≤P CLIQUE

and vice-versa,

CLIQUE ≤P INDEPENDENT-SET


Independent Set

Definition: The complement of a graph G = (V,E) is a graphGc = (V,Ec), whereEc = {(v1, v2)|v1, v2 ∈ V and (v1, v2) 6∈E}.

Claim: If V is an independent set in G, then V is a clique inGc.

’nuff said.


Independent Set

♣Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. – p. 45

Hamiltonian Path

A Hamiltonian path in a (directed or undirected) graph, G,visits each note once.

HAMPATH = {〈G, s, t〉|G has a Hamiltonian path

from s to t}


Hamiltonian Circuit

Hamiltonian Circuit

visits each note once.

ends up where it started


Hamiltonian Circuit

HAMCIRCUIT = {〈G〉 | G has Hamiltonian circuit}

Theorem: HAMPATH is polynomial-time reducible toHAMCIRCUIT,

HAMPATH ≤P HAMCIRCUIT .


Reduction

Theorem: HAMPATH is polynomial-time reducible toHAMCIRCUIT.


Reduction

Theorem: HAMPATH is polynomial-time reducible toHAMCIRCUIT.

Hey, is the new vertex really needed? Why not just add anedge from t to s?


Reduction

Theorem: HAMCIRCUIT is polynomial-time reducible toHAMPATH.

Proof: Left as an easy (recommended) exercise.


Definition

A language B is NP-complete if it satisfies

B ∈ NP , and

Every A in NP is polynomial time reducible to B


Compare

A language B is RE-complete if it satisfies

B ∈ RE, and

Every A in RE is mapping reducible to B


Theorem

Theorem: If B is NP-complete and B ∈ P , then P = NP .

To show P = NP (and make an instant fortune, seewww.claymath.org/millennium/P_vs_NP/), it suffices to find apolynomial-time algorithm for any NP-complete problem.


Theorem

Theorem: If B is NP-complete, C ∈ NP , and B ≤P C, then Cis NP-complete.

We know that C ∈ NP ,

must show that every A in NP is poly-time reducible toC.

Because B is NP-complete,

every language in NP is poly-time reducible to B.

B is poly-time reducible to C

Can compose poly-time reductions (why?), so

A is poly-time reducible to C. ♣


Strategy

Once we have one “structured” NP-complete problem,we can generate more by poly-time reduction.

Getting the first one requires some work.

This is what Steve Cook (then in Berkeley, now inToronto) and Leonid Levin (then in Moscow, now inBoshton) did in the early seventies.


Traveling Salesman

Parameters:

set of cities C

set of inter-city distances D

goal k


Traveling Salesman

Define TRAVELING-SALESMAN = {〈C,D, k〉 | (C,D) has aTS tour of total distance ≤ k}

Remark: Can consider two versions –undirected and directed.

RecallHAMCIRCUIT = {〈G〉 | G has Hamiltonian circuit}

Theorem: HAMCIRCUIT is polynomial-time reducible toTRAVELING-SALESMAN,

HAMCIRCUIT ≤P TRAVELING-SALESMAN .


HAMCIRCUIT≤P TSP

The reduction: Given a directed graph G = (V,E) weconstruct a directed traveling salesman instance.

The cities are identical to the nodes of the originalgraph, C = V .

The distance of going from v1 to v2 is 1 if (v1, v2) ∈ E,and 2 otherwise.

The bound on the total distance of a tour is k = |V |.


HAMCIRCUIT≤P TSP

Validity of Reduction=⇒ Suppose G has a Hamiltonian circuit. Thedistance assigned by the reduction to all edges inthis circuit is 1. Thus in (C,D) there is a travelingsalesman tour of total distance |V | = k, namely(C,D, k) ∈ TRAVELING-SALESMAN.⇐= Suppose (C,D) has a traveling salesman tour oftotal distance |V | = k. Tour cannot contain any edgeof distance 2. Therefore it gives a Hamiltonian circuitin G.

Efficiency: Reduction in quadratic time (filling updistances for all edges of the complete graph). ♣


3SAT (reminder)

Definition: A Boolean formula is in 3CNF form if it is a CNFformula, and all terms have three literals.

(x1 ∨ x2 ∨ x3) ∧ (x3 ∨ x5 ∨ x6) ∧ (x3 ∨ x6 ∨ x4)

Define


Clearly, if φ is a satisfiable 3CNF formula, then for anysatisfying assignment of φ, every clause must contain atleast one literal assigned 1.


The Language SAT

Definition: A Boolean formula is in conjunctive normal form(CNF) if it consists of terms, connected with ∧s.

For example (x1 ∨ x2 ∨ x3 ∨ x4) ∧ (x3 ∨ x5 ∨ x6) ∧ (x3 ∨ x6)

Definition: SAT = {〈φ〉 | φ is satisfiable CNF formula}


Strategy

Once we have one structured NP-complete problem,we can generate more by poly-time reductions.

Getting the first one requires some work.


Cook-Levin (early 70s)

Theorem: SAT is NP complete.

Must show that every NP problem reduces to SAT inpoly-time.

Proof Idea : Suppose L ∈ NP, and M is an NTM thataccepts L.

On input w of length n, M runs in time t(n) = nc.

We consider the nc-by-nc tableau that describes thecomputation of M on input w.


The Tableau

cell[1,1]

cell[1,t(n)]

q 0 0 10 0 . . .1 2 3 . . . t(n)

0 00

1


The Tableau

cell[1,1]

cell[1,t(n)]

q 0 0 10 0 . . .1 2 3 . . . t(n)

0 00

1

Row 1 in tableau represents initial configuration of M oninput w.

Row i in tableau represents i-th configuration in acomputation of M on input w.


A Formula Simulating the Tableau

We construct a Boolean CNF formula φw that “mimics”the tableau.

Given the string w, it takes O(n2c) steps to construct φw.

The following property holds:φw ∈ SAT iff M accepts w.

So the mapping w 7→ φw is a poly time reduction from Lto SAT , establishing L ≤P SAT .

We still got a few small details to take care of...


Details of Formula (Partial List)

We construct a Boolean CNF formula φw that “mimics”the tableau:

φw uses Boolean variables of three types.bi,j,σ is true iff the j-th cell in i-th configurationcontains the letter σ ∈ Γ.si,q is true iff in i-th configuration, M is in state q ∈ Q.hi,j is true iff in i-th configuration M , has is head incell j on tape.

The formula φw consists of four parts:φw = φunique(M)

∧ φstart(w) ∧ φaccept(M) ∧

φcompute(M)


Details of Formula (cont.)

φunique(M)guarantees that the variables encode legal

configurations. For example, at most one of bi,j,0 andbi,j,1 is true.

φstart(w) guarantees that the variables corresponding

to the first row (i = 1) encode the initial configuration ofM on w.

φaccept(M) guarantees that M reached an accepting

configuration.

φcompute(M) guarantees that the configuration

described by the i + 1-st row is a legal succession of theconfiguration described by the i-th row.


Details of Formula (cont.)

φcompute(M) is the “heart” of φw. To construct it, we

employ locality of computations.

To determine contents of tableau entry (i, j) (cell j inconfiguration i), only the contents of three tableauentries (from configuration i − 1),(i − 1, j − 1), (i − 1, j), (i − 1, j + 1), and M ’s table, areneeded.

If head not in area, nothing changes. And and if it is,changes are local and are determined using M .

0 00

1


The Tableau in Perspective

cell[1,1]

cell[1,t(n)]

q 0 0 10 0 . . .1 2 3 . . . t(n)

0 00

1


Correctness of Reduction

All four components of φw can be put in CNF form, soφw itself (∧ of the four) is also in CNF.The transformation w 7→ φw is computable in timeO(n2c).An assignment satisfying φunique(M)

∧ φstart(w)∧

φcompute(M) corresponds to a valid computation of M

on w.An assignment satisfying, in addition φaccept(M),

corresponds to an accepting computation of M on w.Therefore M accepts w iff φw ∈ SAT .

For complete details, consult Sipser or take theComplexity course. ♣


Strategy

We have seen that SAT is NP-complete.

We now reduce SAT to 3SAT.

And then will reduce 3SAT to a bunch of other problemsin NP.

In class and recitation will give in detail just a fewexamples.

Full list contains hundreds or thousands of knownNP-complete problems (from combinatorics, operationresearch, VLSI design, computational geometry,bioinformatics, . . . ).

NP-completeness of new and of old problems is stillestablished these days.


SAT and 3SAT

RecallSAT = {〈φ〉 | φ is a satisfiable CNF formula}


The reduction maps CNF formulae to 3CNF ones “clauseby clause”. A clause with ℓ literals is mapped to ℓ clauses,built on the original literals together with ℓ − 1 new ones.

For example:

(x1 ∨ x2 ∨ x3 ∨ x4 ∨ x8)7−→(x1 ∨ y1) ∧ (y1 ∨ x2 ∨ y2) ∧ (y2 ∨ x3 ∨ y3) ∧(y3 ∨ x4∨y4) ∧ (y4 ∨ x8)


SAT ≤P 3SAT

Consider mapping φ 7→ φ3, e.g. (x1 ∨ x2 ∨ x3 ∨ x4 ∨ x8) 7−→(x1 ∨ y1)∧ (y1 ∨x2 ∨ y2)∧ (y2 ∨x3 ∨ y3)∧ (y3 ∨x4∨y4)∧ (y4 ∨x8)Claim: φ has a satisfying assignment iff φ3 does.Proof sketch: ⇐ An assignment satisfying φ3 cannot “rely”on new literals alone – at least one original literal must besatisfied.⇒ An assignment satisfying φ makes at least one literal perclause happy. In the “φ3 clause” of this literal the newveriable is under no constraints. This enables propagationto a satisfying assignment that “relies” on new vars alone inrest of φ3 clauses.

This establishes validity of the reduction. Since it is inpolynomial time (why?), we get SAT ≤P 3SAT. ♣.


3SAT and Its Poor Cousin

We now know that SAT ≤P 3SAT. Since SAT isNP-complete and 3SAT∈ NP, this proves that 3SAT is itselfNP-complete.

What about the 3SAT ≤P SAT direction?

We now want to examine what happens if we further reducethe number of literals per clause in CNF formulae.

Definition: A Boolean formula is in 2CNF if it is a CNFformula, and all terms have at most two literals. Forexample

(x1 ∨ x2) ∧ (x5 ∨ x6) ∧ (x6 ∨ x4)



Definition:


Betting time: Is 2SAT NP-complete? Is it in P?Or maybe we do not know? . . .



Definition:


Betting time: Is 2SAT NP-complete? Is it in P?Or maybe we do not know? . . .

Well, turns out 2SAT is in P. For details, though, you’llhave to refer to the algorithms course.


Chains of Reductions: NPC Problems

SAT

IntegerProg 3SAT

Clique

IndepSet

VertexCover

SetCover

3ExactCover

Knapsack

Scheduling

3Color HamPath

HamCircuit

TRAVELING-SALESMAN


The classes NP , and coNP. More Problems in NP. Poly...

Documents

Transcript of The classes NP , and coNP. More Problems in NP. Poly...