Optimal Algorithms and Inapproximability Results for Every CSP?
description
Transcript of Optimal Algorithms and Inapproximability Results for Every CSP?
![Page 1: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/1.jpg)
Prasad RaghavendraUniversity of Washington
Seattle
Optimal Algorithms and Inapproximability Results for
Every CSP?
![Page 2: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/2.jpg)
Constraint Satisfaction ProblemA Classic Example : Max-3-SAT
Given a 3-SAT formula,Find an assignment to the variables that satisfies the maximum number of clauses.
))()()(( 145532532321 xxxxxxxxxxxx Equivalently the
largest fraction of clauses
![Page 3: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/3.jpg)
Variables : {x1 , x2 , x3 ,x4 , x5} Constraints : 4 clauses
Constraint Satisfaction Problem
Instance :• Set of variables.• Predicates Pi applied on variables
Find an assignment that satisfies the largest fraction of constraints.
Problem :
Domain : {0,1,.. q-1}Predicates : {P1, P2 , P3 … Pr}
Pi : [q]k -> {0,1}
Max-3-SAT
Domain : {0,1}Predicates :
P1(x,y,z) = x ѵ y ѵ z
))()()(( 145532532321 xxxxxxxxxxxx
![Page 4: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/4.jpg)
Generalized CSP (GCSP)
Replace Predicates by Payoff Functions (bounded real valued)
Problem :
Domain : {0,1,.. q-1}Pay Offs: {P1, P2 , P3 … Pr}
Pi : [q]k -> [-1, 1]Pay Off Functions can be Negative
Can model Minimization Problems like Multiway Cut, Min-Uncut.
Objective :
Find an assignment that maximizes the
Average Payoff
![Page 5: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/5.jpg)
Examples of GCSPs
Max-3-SATMax CutMax Di CutMultiway CutMetric Labelling
0-ExtensionUnique Gamesd- to - 1 GamesLabel CoverHorn Sat
![Page 6: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/6.jpg)
Unique GamesA Special Case
E2LIN mod pGiven a set of linear equations of the form:
Xi – Xj = cij mod p
Find a solution that satisfies the maximum number of equations.
x-y = 11 (mod 17)x-z = 13 (mod 17)
…….
z-w = 15(mod 17)
![Page 7: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/7.jpg)
Unique Games Conjecture [Khot 02]
An Equivalent Version [Khot-Kindler-Mossel-O’Donnell]
For every ε> 0, the following problem is NP-hard for large enough prime p
Given a E2LIN mod p system, distinguish between:• There is an assignment satisfying 1-ε fraction of the equations.• No assignment satisfies more than ε fraction
of equations.
![Page 8: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/8.jpg)
Unique Games Conjecture
A notorious open problem, no general consensus either way.
Hardness Results: No constant factor approximation for unique games. [Feige-Reichman]
Algorithm On (1-Є) satisfiable instances
[Khot 02]
[Trevisan]
[Gupta-Talwar] 1 – O(ε logn)
[Charikar-Makarychev-Makarychev]
[Chlamtac-Makarychev-Makarychev]
[Arora-Khot-Kolla-Steurer-Tulsiani-Vishnoi]
)2/( p)loglog(1 pnO
)log(1 3 nO
))/1log((1 5/12 pO
1log1
![Page 9: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/9.jpg)
Why is UGC important?Problem Best
Approximation Algorithm
NP Hardness Unique Games Hardness
Vertex CoverMax CUTMax 2- SATSPARSEST CUTMax k-CSP
20.878
0.9401
1.360.941
0.95461+ε
20.878
0.9401Every Constant
nlog
kk 2/ kkO 2/2 kkO 2/
UG hardness results are intimately connected to the limitations of Semidefinite Programming
![Page 10: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/10.jpg)
Semidefinite Programming
![Page 11: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/11.jpg)
Max Cut
10
15
3
7
11
Input : a weighted graph G
Find a cut that maximizes the number of crossing edges
![Page 12: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/12.jpg)
Max Cut SDP
Quadratic Program
Variables : x1 , x2 … xn
xi = 1 or -1
Maximize
10
15
3
7
11
1
1
1
-1
-1
-1
-1-1
-1
Eji
jiij xxw),(
2)(4
1
Relax all the xi to be unit vectors instead of {1,-1}. All products are replaced by inner products of vectors
Semidefinite Program
Variables : v1 , v2 … vn
| vi |2 = 1
Maximize
Eji
jiij vvw),(
2||4
1
![Page 13: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/13.jpg)
MaxCut Rounding
v1
v2
v3
v4
v5
Cut the sphere by a random hyperplane, and output the induced graph cut.
- A 0.878 approximation for the problem.
![Page 14: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/14.jpg)
General Boolean 2-CSPs
Total PayOff
In Integral Solutionvi = 1 or -1V0 = 1
Triangle Inequality
![Page 15: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/15.jpg)
2-CSP over {0,..q-1}
Total PayOff
![Page 16: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/16.jpg)
Arbitrary k-ary GCSP
•SDP is similar to the one used by [Karloff-Zwick] Max-3-SAT algorithm.•It is weaker than k-rounds of Lasserre / LS+ heirarchies
![Page 17: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/17.jpg)
Results
![Page 18: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/18.jpg)
Two CurvesIntegrality Gap CurveS(c) = smallest value of the integral solution, given SDP value c.
UGC Hardness CurveU(c) = The best polytime computable solution, assuming UGC given an instance with value c.
0 1Optimum Solution
S(c)
U(c)
Fix a GCSP
If UGC is true:U(c) ≥ S(c)
If UGC is false:U(c) is
meaningless!
![Page 19: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/19.jpg)
UG Hardness Result
Roughly speaking,Assuming UGC, the SDP(I), SDP(II),SDP(III) give best
possible approximation for every CSP
c = SDP ValueS(c) = SDP Integrality GapU(c) = UGC Hardness Curve
Theorem 1:For every constant η > 0, and every GCSP Problem,
U(c) < S(c+ η) + η
0 1Optimum Solution
S(c)
U(c)
U(c)
![Page 20: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/20.jpg)
Consequences
If UGC is true, then adding more constraints does not help for any CSP
Lovasz-Schriver, Lasserre, Sherali-Adams heirarchies do not yield better approximation ratios for any CSP in the worst case.
![Page 21: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/21.jpg)
Efficient Rounding Scheme
Roughly speaking, There is a generic
polytime rounding scheme that is optimal for every CSP, assuming UGC.
Theorem:For every constant η > 0, and every GCSP,there is a polytime rounding scheme that outputs a solution of value U(c-η) – η
c = SDP ValueS(c) = SDP Integrality GapU(c) = UGC Hardness Curve
0 1Optimum Solution
S(c)
U(c)
![Page 22: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/22.jpg)
0 1Optimum Solution
S(c)U(c)
NP-hard
algorithm
If UGC is true, then for every Generalized Constraint Satisfaction Problem :
If UGC is false??
•Hardness result doesn’t make sense.
•How good is the rounding scheme?
![Page 23: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/23.jpg)
Unconditionally Roughly Speaking,For 2-CSPs, the Approximation ratio obtained is at least the red curve S(c)
The rounding scheme achieves the integrality gap of SDP for 2-CSPs (both binary and q-ary cases)
S(c) = SDP Integrality Gap
Theorem: Let A(c) be rounding scheme’s performance on input with SDP value = c. For every constant η > 0
A(c) > S(c- η) - η0 1Optimum Solution
S(c)
![Page 24: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/24.jpg)
As good as the best
SDP(II) and SDP(III) are the strongest SDPs used in approximation algorithms for 2-CSPs
The Generic Algorithm is at least as good as the best known algorithms for 2-CSPs
Examples:
Max Cut [Goemans-Williamson]Max-2-SAT [Lewin-Livnat-Zwick]Unique Games [Charikar-Makarychev-Makarychev]
![Page 25: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/25.jpg)
Computing Integrality Gaps
Theorem: For any η, and any 2-CSP, the curve S(c) can be computed within error η.(Time taken depends on η and domain size q)
0 1Optimum Solution
S(c)
Explicit bounds on the size of an integrality gap instance for any 2-CSP.
![Page 26: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/26.jpg)
Related WorkProblem Best
Approximation Algorithm
Unique Games Hardness
Vertex CoverMax CUTMax 2- SATSPARSEST CUTMax k-CSP
20.878
0.9401
2 [Khot-Regev] 0.878 [Khot-Kindler-Mossel-O’donnell]0.9401 [Per Austrin]Every Constant [Chawla-Krauthgamer-..] [Trevisan-Samorodnitsky]
kk 2/ kkO 2/nlog
[Austrin 07]Assuming UGC, and a certain additional conjecture:
``For every boolean 2-CSP, the best approximation is given by SDP(III)”
[O’Donnell-Wu 08]Obtain matching approximation algorithm, UGC hardness and SDP gaps for MaxCut
![Page 27: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/27.jpg)
Proof Overview
![Page 28: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/28.jpg)
Dictatorship TestGiven a function F : {-1,1}R {-1,1}•Toss random coins•Make a few queries to F •Output either ACCEPT or REJECT
F is a dictator functionF(x1 ,… xR) = xi
F is far from every dictator function
(No influential coordinate)
Pr[ACCEPT ] = Completeness
Pr[ACCEPT ] =Soundness
![Page 29: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/29.jpg)
ConnectionsSDP Gap Instance
SDP = 0.9OPT = 0.7
UG Hardness
0.9 vs 0.7
Dictatorship Test
Completeness = 0.9Soundness = 0.7
[Khot-Kindler-Mossel-O’Donnell]
[Khot-Vishnoi]For sparsest cut, max cut.[This Paper]
All these conversions hold for every GCSP
![Page 30: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/30.jpg)
A Dictatorship Test for Maxcut
CompletenessValue of Dictator Cuts
F(x) = xi
SoundnessThe maximum value attained by a cut far from a dictator
A dictatorship test is a graph G on the hypercube.A cut gives a function F on the hypercube
Hypercube = {-1,1}100
![Page 31: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/31.jpg)
An Example : Maxcutv1
v2
v3
v4
v5
10
15
3
71
1
100 dimensional hypercube
Graph G SDP Solution
CompletenessValue of Dictator Cuts =
SDP Value (G)
SoundnessGiven a cut far from every dictator :It gives a cut on graph G with the same value.
In other words, Soundness ≤ OPT(G)
![Page 32: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/32.jpg)
From Graphs to Tests10
15
3
7
11
v1
v2
v3
v4
v5
Graph G (n vertices)
100 dimensional hypercube : {-1,1}100
SDP Solution
For each edge e, connect every pair of vertices in hypercube separated by the length of e
Constant independent of
size of G
H
![Page 33: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/33.jpg)
Completeness
Echoice of edge e=(u,v) in G
[EX,Y in 100 dim hypercube with dist |u-v|^2 [ (F(X)-F(Y))2 ] ]
v1
v2
v3
v4
v5
100 dimensional hypercube
-1
-1-1
1
1
1
For each edge e, connect every pair of vertices in hypercube separated by the length of e
Set F(X) = X1
(X1 – Y1)2
X1 is not equal to Y1 with probability |u-v|2 , hence completeness = SDP Value (G)
![Page 34: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/34.jpg)
The Invariance Principle
Invariance Principle for Low Degree Polynomials[Rotar] [Mossel-O’Donnell-Oleszkiewich], [Mossel 2008]
“If a low degree polynomial F has no influential coordinate, then F({-1,1}n) and F(Gaussian) have similar distribution.”
A generalization of the following fact :
``Sum of large number of {-1,1} random variableshas similar distribution as
Sum of large number of Gaussian random variables.”
![Page 35: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/35.jpg)
From Hypercube to the Sphere
100Dimensional hypercube
100 dimensio
nal sphere
F : [-1,1]
Express F as a multilinear polynomial using Fourier expansion, thus extending it to the sphere.
P : Real numbers
Since F is far from a dictator, by invariance principle, its behaviour on the sphere is similar to its behaviour on hypercube.
Nearly always [-1,1]
![Page 36: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/36.jpg)
A Graph on the Sphere10
15
3
7
11
v1
v2
v3
v4
v5
Graph G (n vertices)
100 dimensional sphere
SDP Solution
For each edge e, connect every pair of vertices in sphere separated by the length of e
S
![Page 37: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/37.jpg)
Hypercube vs Sphere
H S
F:{-1,1}100 -> {-1,1} is a cut far from every dictator.
P : sphere -> Nearly {-1,1}Is the multilinear extension of F
By Invariance Principle,
MaxCut value of F on H ≈ Maxcut value of P on S.
![Page 38: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/38.jpg)
Soundnessv1
v2
v3
v4
v5 For each edge e in the graph G connect every pair of vertices in hypercube separated by the length of e
SG
Alternatively, generate S as follows:Take the union of all possible rotations of the graph G
S consists of union of disjoint copies of G. Thus, MaxCut Value of S < Max cut value of G.
Hence MaxCut value of F on H is at most the max cut value of G. Soundness ≤ MaxCut(G)
![Page 39: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/39.jpg)
Algorithmically,
Given a cut F of the hypercube graph H• Extend F to a function P on the sphere using
its Fourier expansion.• Pick a random rotation of the SDP solution to
the graph G• This gives a random copy Gc of G inside the
sphere graph S• Output the solution assigned by P to GC
![Page 40: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/40.jpg)
Roughly FormallySample R Random Directions
Sample R independent vectors : g(1), g(2) ,.. g(100) Each with i.i.d Gaussian components.
Project along them
Project each vi along all directions g(1), g(2) ,.. g(100)
Yi(j) = v0 v∙ i + (1-ε)(vi – (v0 v∙ i)v0) g∙ (100)
Compute P on projections
Compute xi = P(Yi
(1) , Yi(2) ,.. Yi
(100))Round the output of P
If xi > 1, xi = 1 If xi < -1, xi = -1 If xi is in [-1,1]
xi = 1 with probability (1+xi)/2 -1 with probability (1-xi)/2
Given the Polynomial P(y1,… y100)
![Page 41: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/41.jpg)
Key Lemma
Any CSP Instance
G
DICTGDictatorship Test
on functionsF : {-1,1}n ->{-1,1}
If F is far from a dictator,RoundF (G) ≈ DICTG (F)
1) Tests of the verifier are same as the constraints in instance G2) Completeness = SDP(G)
Any Function
F: {-1,1}n → {-1,1}
RoundFRounding Schemeon CSP Instances G
![Page 42: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/42.jpg)
UG Hardness Result
InstanceSDP = cOPT = s
Dictatorship Test
Completeness = cSoundness <= s
UG Hardness
Completeness = cSoundness <= s
Worst Case Gap Instance
Theorem 1:For every constant η > 0, and every GCSP Problem,
U(c) < S(c+ η) + η
![Page 43: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/43.jpg)
Generic Rounding Scheme
Solve SDP(III) to obtain vectors (v1 ,v2 ,… vn )
Add little noise to SDP solution (v1 ,v2 ,… vn )
For all multlinear polynomials P(y1 ,y2, .. y100) do
Round using P(y1 ,y2, .. y100)
Output the best solution obtained
P is Multilinear polynomial in 100 variables with coefficients in [-1,1]
![Page 44: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/44.jpg)
Algorithm
Instance ISDP = cOPT = ?
AnyDictatorship
TestCompleteness = c
UG Hardness
Completeness = c
Soundness of any Dictatorship Test ≥ U(c)
There is some function F : {0,1}R -> {0,1} that hasPr[F is accepted] ≥ U(c)
By Key Lemma, Performance of F as rounding polynomial on instance I = Pr[F is accepted] > U(c)
Dictatorship Test (I)
Completeness = c
![Page 45: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/45.jpg)
Related Developments
• Multiway Cut and Metric Labelling problems.
• Maximum Acyclic Subgraph problem
• Bipartite Quadratic Optimization Problem (Computing the Grothendieck constant)
[Manokaran, Naor, Schwartz, Raghavendra]
[Guruswami,Manokaran, Raghavendra]
[Raghavendra,Steurer]
![Page 46: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/46.jpg)
Conclusions
Unique Games and Invariance Principle connect : Integrality Gaps, Hardness Results ,Dictatorship tests and Rounding Algorithms.
These connections lead to new algorithms, and hardness results unifying several known results.
![Page 47: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/47.jpg)
Thank You
![Page 48: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/48.jpg)
Rounding Scheme(For Boolean CSPs)
Rounding Scheme was discovered by the reversing the soundness analysis.This fact was independently observed by Yi Wu
![Page 49: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/49.jpg)
MaxCut Rounding
v1
v2
v3
v4
v5
Cut the sphere by a random hyperplane, and output the induced graph cut.
Equivalently,
•Pick a random direction g.•For each vector vi , project vi along g
yi = vi . g•Assign
xi = 1 if yi > 0xi = 0 otherwise.
![Page 50: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/50.jpg)
SDP Rounding Schemes
SDP Vectors (v1 , v2 .. vn )
Projections(y1 , y2 .. yn )
Assignment
Random Projection
Process the projections
For any CSP, it is enough to do the following:
Instead of one random projection, pick sufficiently many (say 100) projections
Use a multi linear polynomial P to process the projections
![Page 51: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/51.jpg)
UG Hardness Results
InstanceSDP = cOPT = s
Dictatorship Test
Completeness = cSoundness <= s
UG Hardness
Completeness = cSoundness <= s
Worst Case Gap Instance
Theorem 1:For every constant η > 0, and every GCSP Problem,
U(c) < S(c+ η) + η
![Page 52: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/52.jpg)
Multiway Cut and Labelling Problems
Theorem: Assuming Unique Games Conjecture,The earthmover linear program gives the best approximation.
Theorem: Unconditionally, the simple SDP does not give better approximations than the LP.
10
15
3
7
11
3-Way Cut:Separate the 3-terminals while
separating the minimum number of edges
[Manokaran, Naor, Schwartz, Raghavendra]
![Page 53: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/53.jpg)
Maximum Acyclic Subgraph
Given a directed graph, order the vertices to maximize the number of forward edges.
[Guruswami,Manokaran, Raghavendra]
Theorem: Assuming Unique Games Conjecture,The best algorithm’s output is as good as a random ordering.
Theorem: Unconditionally, the simple SDP does not give better approximations than random.
![Page 54: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/54.jpg)
The Grothendieck Constant
The Grothendieck constant is the smallest constant k(H) for which the following inequality holds for all matrices :
The constant is just the integrality gap of the SDP for bipartite quadratic optimization.
Value of the constant is between 1.6 and 1.7 but is unknown yet.
[Raghavendra,Steurer]
![Page 55: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/55.jpg)
Grothendieck Constant[Raghavendra,Steurer]
Theorem: There is an algorithm to compute arbitrarily good approximations to the Grothendieck constant.
Theorem: There is an efficient algorithm that solves the bipartite quadratic optimization problem to an approximation equal to Grothendieck constant.
![Page 56: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/56.jpg)
If all this looks deceptively simple, then it is because there was deception
Working with several probability distributions at once.
![Page 57: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/57.jpg)
UG Hardness Results
InstanceSDP = cOPT = s
Dictatorship Test
Completeness = cSoundness <= s
UG Hardness
Completeness = cSoundness <= s
Worst Case Gap Instance
Best UG Hardness =
Integrality GapU(c) < S(c+η) + η
![Page 58: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/58.jpg)
Algorithm
Instance ISDP = cOPT = ?
AnyDictatorship
TestCompleteness = c
UG Hardness
Completeness = c
Soundness of any Dictatorship Test ≥ U(c)
There is some function F : {0,1}R -> {0,1} that hasPr[F is accepted] ≥ U(c)
By Key Lemma, Performance of F as rounding polynomial on instance I = Pr[F is accepted] > U(c)
Dictatorship Test (I)
Completeness = c
![Page 59: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/59.jpg)
On some instance I with SDP value = c , algorithm outputs a solution with value s.
For every function F far from dictator ,
Performance of F in rounding I ≤ s
By Key Lemma, For every such F
Pr[ F is accepted by Dict(I) ] ≤ s
Thus the Dict(I) is a test with soundness s.
Unconditional Results For 2-CSPs
![Page 60: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/60.jpg)
Unconditional Results For 2-CSPs
Dictatorship Test(I)
Completeness = cSoundness = s
UG Hardness
Completeness = cSoundness = s
UG Integrality Gap instance
Integrality Gap instanceSDP = cOPT ≤ sAlgorithm’s performance
matches the integrality gap of the SDP
[Khot-Vishnoi]
![Page 61: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/61.jpg)
Computing Integrality Gaps
Integrality gap of a SDP relaxation = Worst case ratio of Integral Optimum
SDP Optimum
Worst Case over all instances - an infinite set
Due to tight relation of integrality gaps/ dictatorship tests for 2-CSPs
Integrality gap of a SDP relaxation = Worst case ratio of Soundness
CompletenessThis time the worst case is along all dictatorship tests on {-1,1}R
- a finite set that can be discretized.
![Page 62: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/62.jpg)
Key Lemma : Through An Example
1
2132
322
21 ||||||3
1vvvvvv
SDP:Variables : v1 , v2 ,v3
|v1|2 = |v2|2 = |v3|2 =1
Maximize2 3
![Page 63: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/63.jpg)
E[a1 a2] = v1 v∙ 2
E[a12] = |v1|2 E[a2
2] = |v2|2
For every edge, there is a local distribution over integral solutions such that:All the moments of order at most 2 match the inner products.
Local Random Variables
1
32
Fix an edge e = (1,2).
There exists random variables a1 a2 taking values {-1,1} such that:
c = SDP Valuev1 , v2 , v3 = SDP Vectors
A12A13
A23
![Page 64: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/64.jpg)
Dictatorship TestPick an edge (i,j)Generate ai,aj in {-1,1}R as follows:The kth coordinates aik ,ajk come from distribution Aij
Add noise to ai,aj
Accept if F(ai) ≠ F(aj)
c = SDP Valuev1 , v2 , v3 = SDP Vectors
A12,A23,A31 = Local Distributions
1
32
A12
Input Function:F : {-1,1}R -> {-1,1}
Max Cut Instance
![Page 65: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/65.jpg)
AnalysisPick an edge (i,j)Generate ai,aj in {-1,1}R as follows:
The kth coordinates aik,ajk come from distribution Aij
Add noise to ai,aj
Accept if F(ai) ≠ F(aj)
A12,A23,A31 = Local Distributions
1
32
Max Cut Instance
]))()([(
4
1]))()([(
4
1]))()([(
4
1
3
1 213
232
221 312312
aFaFEaFaFEaFaFE AAA
Input Function:F : {-1,1}R -> {-1,1}
![Page 66: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/66.jpg)
]))()([(
4
1]))()([(
4
1]))()([(
4
1
3
1 213
232
221 312312
aFaFEaFaFEaFaFE AAA
CompletenessA12,A23,A31 = Local Distributions
Input Function is a Dictator : F(x) = x1
])[(
4
1])[(
4
1])[(
4
1
3
1 21131
23121
22111 312312
aaEaaEaaE AAA
Suppose (a1 ,a2) is sampled from A12 then :E[a11 a21] = v1 v∙ 2 E[a11
2] = |v1|2 E[a212] = |v2|2
221
221 ||])[(
12vvaaEA
Summing up, Pr[Accept] = SDP Value(v1 , v2 ,v3)
![Page 67: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/67.jpg)
E[b1 b2] = v1 v∙ 2 E[b2 b3] = v2 v∙ 3 E[b3 b1] = v3 v∙ 1
E[b1
2] = |v1|2 E[b22] = |v2|2 E[b3
2] = |v3|2
There is a global distribution B=(b1 ,b2 ,b3) over real numbers such that:All the moments of order at most 2 match the inner products.
Global Random Variablesc = SDP Value
v1 , v2 , v3 = SDP Vectors
g = random Gaussian vector.(each coordinate generated by i.i.d normal variable)
b1 = v1 g∙b2 = v2 g∙b3 = v3 g∙
1
32
B
![Page 68: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/68.jpg)
Rounding with Polynomials
Input Polynomial : F(x1 ,x2 ,.. xR)
Generate b1 = (b11 ,b12 ,… b1R)
b2 = (b21 ,b22 ,… b2R)
b3 = (b31 ,b32 ,… b3R)
with each coordinate (b1t ,b2t ,b3t) according to global distribution B
Compute F(b1),F(b2) ,F(b3)
Round F(b1),F(b2),F(b3) to {-1,1}
Output the rounded solution.
1
32
B
]))()([(
4
1]))()([(
4
1]))()([(
4
1
3
1 213
232
221 bFbFEbFbFEbFbFE BBB
![Page 69: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/69.jpg)
Invariance
Suppose F is far from every dictator then since A12 and B have same first two moments,
F(a1),F(a2) has nearly same distribution as F(b1),F(b2)
•
• F(b1), F(b2) are close to {-1,1}
]))()([(4
1]))()([(
4
1 221
22112
bFbFEaFaFE BA
![Page 70: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/70.jpg)
From Gap instances to Gap instances
InstanceSDP = cOPT = s
Dictatorship Test
Completeness = cSoundness = s
UG Hardness
Completeness = cSoundness = s
UG Gap instance for a
Strong SDP
A Gap Instance for the Strong SDP for
CSP
![Page 71: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/71.jpg)
For each variable u in CSP,Introduce q variables : {u0 , u1 ,.. uq-1 }
uc = 1,
ui = 0 for i≠c
Payoff for u,v :P(u,v) = ∑a ∑b P(a,b)ua vb
2-CSP over {0,..q-1}
u = c
![Page 72: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/72.jpg)
2-CSP over {0,..q-1}
Total PayOff
![Page 73: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/73.jpg)
Arbitrary k-ary GCSP
SDP is similar to the one obtained by k-rounds of Lasserre
![Page 74: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/74.jpg)
Rounding Scheme(For Boolean CSPs)
Rounding Scheme was discovered by the reversing the soundness analysis.This fact was independently observed by Yi Wu
![Page 75: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/75.jpg)
SDP Rounding Schemes
SDP Vectors (v1 , v2 .. vn )
Projections(y1 , y2 .. yn )
Assignment
Random Projection
Process the projections
For any CSP, it is enough to do the following:
Instead of one random projection, pick sufficiently many projections
Use a multilinear polynomial P to process the projections
![Page 76: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/76.jpg)
Roughly FormallySample R Random Directions
Sample R independent vectors : w(1), w(2) ,.. w(R) Each with i.i.d Gaussian components.
Project along them
Project each vi along all directions w(1), w(2) ,.. w(R)
Yi(j) = v0 v∙ i + (1-ε)(vi – (v0 v∙ i)v0) w∙ (j)
Compute P on projections
Compute xi = P(Yi
(1) , Yi(2) ,.. Yi
(R))Round the output of P
If xi > 1, xi = 1 If xi < -1, xi = -1 If xi is in [-1,1]
xi = 1 with probability (1+xi)/2 -1 with probability (1-xi)/2
Rounding By Polynomial P(y1,… yR)
![Page 77: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/77.jpg)
Algorithm
Solve SDP(III) to obtain vectors (v1 ,v2 ,… vn )
Smoothen the SDP solution (v1 ,v2 ,… vn )
For all multlinear polynomials P(y1 ,y2, .. yR) do
Round using P(y1 ,y2, .. yR)
Output the best solution obtained
R is a constant parameter
![Page 78: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/78.jpg)
“For all multilinear polynomials P(y1 ,y2, .. yR) do”
- All multilinear polynomials with coefficients bounded within [-1,1]- Discretize the set of all such multi-linear polynomials
There are at most a constant number of such polynomials.
Discretization
![Page 79: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/79.jpg)
Smoothening SDP Vectors
Let u1 ,u2 .. un denote the SDP vectors corresponding to the following distribution over integral solutions:``Assign each variable uniformly and independently at random”
Substitute vi
* v∙ j* = (1-ε) (vi v∙ j) + ε (ui u∙ j)
![Page 80: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/80.jpg)
Non-Boolean CSPs
There will be q rounding polynomials instead of one polynomial.
Projection is in the same fashion: Yi
(j) = v0 v∙ i + (1-ε)(vi – (v0 v∙ i)v0) w∙ (j)
To Round the Output of the polynomial, do the following:
![Page 81: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/81.jpg)
From Gap instances to Gap instances
InstanceSDP = cOPT = s
Dictatorship Test
Completeness = cSoundness = s
UG Hardness
Completeness = cSoundness = s
UG Gap instance for a
Strong SDP
A Gap Instance for the Strong SDP for
CSPWorst Case
Instance
![Page 82: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/82.jpg)
Backup Slides
![Page 83: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/83.jpg)
Rounding for larger domains
![Page 84: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/84.jpg)
Remarks
For every CSP and every ε > 0, there is a large enough constant R such that
• Approximation achieved is within ε of optimal for all CSPs if Unique Games Conjecture is true.
• For 2-CSPs, the approximation ratio is within ε of the integrality gap of the SDP(I).
![Page 85: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/85.jpg)
Rounding Schemes
Very different rounding schemes for every CSP.with often complex analysis.
Max Cut - Random hyperplane cutting Multiway cut - Complicated Cutting the simplex.
• Our algorithm is a generic rounding procedure.• Analysis does not compute the approximation
factor, but indirectly shows that it is equal to the integrality gap.
![Page 86: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/86.jpg)
“Sample R independent vectors : w1, w2 ,.. wR each with i.i.d Gaussian components.For all multlinear polynomials P(y1 ,y2, .. yR) do
Compute xi = P(vi w∙ 1 , vi w∙ 2 ,.. vi w∙ R)”
Goemans-Williamson rounding uses one single random projection, this algorithm uses a constant number of random projections.
![Page 87: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/87.jpg)
Semidefinite Programming
• Linear program over the inner products• Strongest algorithmic tool in approximation
algorithms• Used in a large number of algorithms.
Integrality gap of a SDP relaxation = Worst case ratio of Integral Optimum
SDP Optimum
![Page 88: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/88.jpg)
More Constraints?
Most SDP algorithms use simple relaxations with few constraints.
[Arora-Rao-Vazirani] used the triangle inequalities to get sqrt(log n) approximation for sparsest cut.
Can the stronger SDPs yield better approximation ratios for problems of interest?
![Page 89: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/89.jpg)
Max Cut
10
15
3
7
11
Input : a weighted graph G
Find a cut that maximizes the number of crossing edges
![Page 90: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/90.jpg)
Max Cut SDP
Quadratic Program
Variables : x1 , x2 … xn
xi = 1 or -1
Maximize
10
15
3
7
11
1
1
1
-1
-1
-1
-1-1
-1
Eji
jiij xxw),(
2)(4
1
Relax all the xi to be unit vectors instead of {1,-1}. All products are replaced by inner products of vectors
Semidefinite Program
Variables : v1 , v2 … vn
| vi |2 = 1
Maximize
Eji
jiij vvw),(
2||4
1
![Page 91: Optimal Algorithms and Inapproximability Results for Every CSP?](https://reader036.fdocuments.in/reader036/viewer/2022062323/56815b28550346895dc8e6c3/html5/thumbnails/91.jpg)
Semidefinite Program
Variables : v1 , v2 … vn
| vi |2 = 1
Maximize
Max Cut SDP
10
15
3
7
11
1
1
1
-1
-1
-1
-1-1
-1
Eji
jiij vvw),(
2||4
1