Constraint Satisfaction over a Non-Boolean Domain

Approximation Algorithms and Unique Games Hardness

Venkatesan Guruswami Prasad RaghavendraUniversity of Washington

Seattle, WA

Constraint Satisfaction ProblemA Classic Example : Max-3-SAT

Given a 3-SAT formula,Find an assignment to the variables that satisfies the maximum number of clauses.

))()()(( 145532532321 xxxxxxxxxxxx Equivalently the

largest fraction of clauses

Constraint Satisfaction Problem

General Definition :

Domain : {0,1,.. q-1}Predicates : {P1, P2 , P3 … Pr}

Pi : [q]k -> {0,1}

Arity : Maximum number of variables per constraint (k)

Example : Max-3-SAT

Domain : {0,1}Predicates : P1(x,y,z) = x ѵ y ѵ z

Arity = 3

GOAL : Find an assignment satisfying maximum fraction of constraints

Approximability

Max-3-SAT : 7/8 [Karloff-Zwick],[Hastad]

Most Max-CSP problems are NP-hard to solve exactly.

Different Max-CSP problems are approximable to varying ratios.

3-XOR : ½[Hastad]

Max-Cut : 0.878[Goemans-Williamson],

[Khot-Kindler-Mossel-O’donnel]

Max-2-SAT : 0.94 [Lempel-Livnat-Zwick],

[Austrin]

Question :Which Max-CSP is the hardest to approximate?

Refined Question :Among all Max-CSP problems over domain [q] ={0,..q-1}, and arity k, which is the hardest to approximate?

Clearly, the problems become harder as domain size or the arity grows

PCP Motivation

Probabilistically Checkable Proof

(A string over alphabet {0,1,..q-1})

))()()(( 145532532321 xxxxxxxxxxxx

Verifier

Random bits

ACCEPT/REJECT

Completeness(C) :

If SAT formula is satisfiable, there is a proof that verifier accepts with probability C

Soundness (S):

For an unsatisfiable formula, no proof is accepted with probability more than S

Among all Max-CSP problems over domain [q] ={0,..q-1}, and arity k, which is the hardest to approximate?

PCP Motivation

What is the best possible gap between completeness (c) and soundness (s) for a PCP verifier that makes k queries over an alphabet [q] = {0,1,..q-1} ?

Boolean CSPs

k

k

2

2 2

k

k

2

2

Hardness:For every k, there is a boolean CSP of arity k, which is NP-hard to approximate better than :

Algorithm:Every boolean CSP of arity k, can be approximated to a factor :

[Samorodnitsky-Trevisan 2000]simplified by [Hastad-Wigderson] k

k

2

22

[Engebresten-Holmerin]

[Samorodnitsky-Trevisan 2006]

Assuming Unique Games Conjecture

k2

1Random Assignment

k2

2[Trevisan]

k

kk

2

log/[Hast]

k

k

2

88.0[Charikar-Makarychev-Makarychev]

This Work : Non-Boolean CSPsUG Hardness: Assuming Unique Games Conjecture, For every k, and a prime number q, there is a CSP of arity k over the domain [q] ={0,1,2,..q-1}, which is NP-hard to approximate better than

Algorithm: The algorithm of [Charikar-Makarychev-Makarychev] can be extended to non-boolean domains.

Every CSP of arity k over the domain [q] ={0,1,2,..q-1} can be approximated to a factor

kq

kq2

kq

qk 7/

Related Work[Raghavendra 08] “Optimal approximation algorithms and

hardness results for every CSP, assuming Unique Games Conjecture.” – “Every” so applies to the hardest CSPs too.– Does not give explicit example of hardest CSP, nor the explicit value of the

approximation ratio.

[Austrin-Mossel 08] “Assuming Unique Games conjecture, For every prime power q, and k, it is NP-hard to

approximate a certain CSP over [q] to a factor > ”

– Independent work using entirely different techniques(invariance principle)– Show a more general result, that yields a criteria for Approximation Resistance

of a predicate.

kq

kqq )1(

Techniques

• We extend the proof techniques of [Samorodnitsky-Trevisan 2006] to non- boolean domains.

• To this end, we – Define a subspace linearity test.– Show a technical lemma relating the success probability of a

function F to the Gower’s norm of F (similar to the standard proof relating the number of

multidimensional arithmetic progressions to the Gower’s norm)

• Along the way, we make some minor simplifications to [Samorodnitsky-Trevisan 2006]. – (Remove the need for common influences)

Proof Overview

Given a function F : [q]R [q], • Make at most k queries to F• Based on values of F, Output ACCEPT or REJECT. Distinguish between the following two cases :

Dictatorship Testing Problem

F is a dictator functionF(x1 ,… xR) = xi

F is far from every dictator function

(No influential coordinate)

Pr[ACCEPT ] = Completeness

Pr[ACCEPT ] =Soundness

Goal : Achieve maximum gap between Completeness and Soundness

UG Hardness Proofs

UG Hardness Result:Assuming Unique Games Conjecture, it is NP-hard to approximate a CSP over [q] with

arity k to ratio better than C/S

Using [Khot-Kindler-Mossel-O’Donnell] reduction.

Dictatorship Test Over functions F:[q]R -> [q]

Completeness = C Soundness = S

# of queries = k

For the rest of the talk, we shall focus

on Dictatorship Testing.

Testing Dictatorships by Testing Linearity[Samorodnitsky-Trevisan 2006]

Fix {0,1} : field on 2 elements k = 2d

Given a functionF : {0,1}R -> {0,1}

• Pick a random affine subspace A of dimension d.

• Test if F agrees with some affine linear function on the subspace A.

Every dictator F(x1 , x2 ,.. xR ) = xi is a linear function over vector space

{0,1}R

Random Assignment :

There are 2d+1 different affine linear functions on A.

There are possible functions on A.

So a random function satisfies the test with probability

d22

k

d kd

2

2

2

22

1

Gower’s Norm

For F : {0,1}R -> {0,1}, let f(x) = (-1)F(x) .

dth Gowers Norm Ud(f) =

E [ product of f over C]Expectation over random d-dimensional subcubes C in {0,1}R

},..1{| dSyxCSi

i

x x+y1

x+y2 x+y1+y2

x x+y1

x+y2 x+y1+y2

x x+y1

x+y3+ y2 x+y1+y2+y3

x+y3 x+y1+y3

d-dimensional cube spanned by {x,y1 ,y2 ,.. yd } is

Cubes

More Formally,

Intuitively, the dth Gower’s norm measures the correlation of the function f with degree d-1 polynomials.

Gower’s Norm

Testing Dictatorships by Testing Linearity[Samorodnitsky-Trevisan 2006]

Lemma : If F : {0,1}R -> {0,1} passes the test with probability then f = (-1)F has high dth Gowers Norm. (k=2d)

k

k

2

2

Lemma : If a balanced function f : {0,1}R -> {-1,1} has high dth Gowers Norm, then it has an influential coordinate (k=2d)

Theorem : If a balanced function F : {0,1}R -> {0,1} passes the test with probability then it has an influential coordinate

k

k

2

2

Using Noise sensitivity,There are only a FEW

influential coordinates.

Extending to Larger domains

Fix [q]: field on q elements(q is a prime). k = qd

Given a functionF : [q]R -> [q]

• Pick a random affine subspace A of dimension d.

• Test if F agrees with some affine linear function on the subspace A.

Replace 2 by q in the [Samorodnitsky-

Trevisan] dictatorship test.

The Difficulty

Lemma : If F : {0,1}R -> {0,1} passes the test with probability then f = (-1)F has high dth Gowers Norm. (k=2d)

k

k

2

2

Over {0,1}R,

Subcube = Affine subspace.

Testing linearity over a random affine subspace, can be easily related to expectation over a random cube.

Over [q]R , Subcube ≠ Affine subspace. (2R points) (qR points)

Success probability of a function F : {0,1}R -> {0,1}, is related to :let f(x) = (-1)F(x) .

E [ product of f over A]

Expectation over random d-dimensional affine subspace A in [q]R

(Affine subspaces are like multidimensional arithmetic progressions)

x x+y1 x+2y1 x+(q-2)y1x+(q-1)y1

x+y2+y1x+y2+2y1 x+y2+ (q-2)y1 x+y2+ (q-1)y1

x x+y1 x+2y1x+ (q-2)y1 x+ (q-1)y1

x+2y2+y1x+2y2+2y1 x+2y2+ (q-2)y1 x+2y2+ (q-1)y1

x+(q-2)y2+y1 x+(q-2)y2+2y1x+(q-2)y2+ (q-2)y1x+(q-2)y2+ (q-1)y1x+(q-2)y2

x+2y2

x+y2

x+(q-1)y2x+(q-1)y2+y1 x+(q-1)y2+2y1x+(q-1)y2+ (q-2)y1x+(q-1)y2+ (q-1)y1

Multidimensional Progressions

E [ product of f over C]

Expectation over random dq-dimensional subcubes C in [q]R

Alternate Lemma

Lemma : If F : [q]R -> [q] passes the test with probability then f = (-1)F has high dqth Gower’s Norm. (k=qd)

kq

kq2

• d-dimensional affine subspace test relates to the dqth Gower’s norm

• The proof is technical and involves repeated use of the Cauchy-Schwartz inequality.

• A special case of a more general result by [Green-Tao][Gowers-Wolf], where they define

“Cauchy-Schwartz Complexity” of a set of linear forms.

Open Questions

CSPs with Perfect Completeness:Which CSP is hardest to approximate, under the promise that the input instance is completely satisfiable?

Approximation Resistance:Characterize CSPs for which the best approximation achievable is given by a random assignment.

Thank You

Unique GamesA Special Case

E2LIN mod pGiven a set of linear equations of the form:

Xi – Xj = cij mod p

Find a solution that satisfies the maximum number of equations.

x-y = 11 (mod 17)x-z = 13 (mod 17)

…….

z-w = 15(mod 17)

Unique Games Conjecture [Khot 02]

An Equivalent Version [Khot-Kindler-Mossel-O’Donnell]

For every ε> 0, the following problem is NP-hard for large enough prime p

Given a E2LIN mod p system, distinguish between:• There is an assignment satisfying 1-ε fraction

of the equations.• No assignment satisfies more than ε fraction

of equations.