An Algorithmic Proof of the Lopsided Lovasz Local Lemma Nick Harvey University of British Columbia...
-
Upload
clyde-dawson -
Category
Documents
-
view
217 -
download
1
Transcript of An Algorithmic Proof of the Lopsided Lovasz Local Lemma Nick Harvey University of British Columbia...
An Algorithmic Proof of the Lopsided Lovasz Local
Lemma
Nick Harvey University of British Columbia
Jan VondrakIBM Almaden
• Let E1,E2,…,En be events in a discrete probability space.
• Let G be a graph on {1,…,n}.Let ¡(i) be neighbors of i, and let ¡+(i) = ¡(i) [ {i}.
• G is called a dependency graph if, for all i,Ei is jointly independent of { Ej : j¡+(i) }.
• Erdos-Lovasz ‘75: Suppose |¡+(i)|· d and P[Ei] · p 8i.If ped· 1 then P[Åi Ei
c] ¸ (1-1/d)n.
Lovasz Local Lemma
E1
E2
E3
E4
E.g., Pairwise Independent Events
Example: k-SAT• Let Á be a k-SAT formula• Pick a uniformly random assignment to the
variables.• Let Ei be the event clause i is unsatisfied. Note
P[Ei]=2-k=p.
• Suppose that each variable appears in 2k/ek clauses.
• Dependency graph: |¡+(i)|· 2k/e = d.• Eg:
• Since ped· 1, LLL implies P[Åi Eic]>0, so Á is
satisfiable.
E1
Clause 1 unsatisfiedE2
Clause 2 unsatisfied
E3
Clause 3 unsatisfiedE4
Clause 4 unsatisfied
Á = (x1Çx2Çx3) Æ (x1Çx4Çx5) Æ (x4Çx5Çx6) Æ
(x2Çx3Çx6)
Stronger forms of LLL
• Symmetric LLL: P[Ei] · 1/(e¢maxj |¡+
(j)|) 8i.• Asymmetric LLL: j2¡+(i) P[Ej] · 1/4
8i.• General LLL: 9 x1,...,xn 2 (0,1)
such thatP[Ei] · xi ¢ j2¡(i) (1-xj) 8i.
• Symmetric LLL: P[Ei] · 1/(e¢maxj |¡+
(j)|) 8i.• Asymmetric LLL: j2¡+(i) P[Ej] · 1/4
8i.• General LLL: 9 y1,...,yn > 0 such
thatP[Ei] · yi / j2¡+(i) (1+yj) 8i.
Stronger forms of LLL
• Symmetric LLL: P[Ei] · 1/(e¢maxj |¡+
(j)|) 8i.• Asymmetric LLL: j2¡+(i) P[Ej] · 1/4
8i.• General LLL: 9 y1,...,yn > 0 such
thatP[Ei] · yi / Jµ¡+(i) yJ 8i.
where yJ = j2J yj.
• Let Ind be the independent sets of the dependency graph.
• Cluster Expansion: 9 y1,...,yn > 0 such that
P[Ei] · yi / Jµ¡+(i) yJ 8i.
J 2 Ind
Stronger forms of LLL
• Let Ind be the independent sets of the dependency graph.
• Let pi = P[Ei].
• For SµV, let
• Shearer ‘85: If qI>0 8I2Ind then P[Åi Eic]
¸ q; > 0.Moreover, for every dependency graph, this is theoptimal criterion under which the LLL is true.
“multivariateindependencepolynomial”
Stronger forms of LLL
Algorithmic Results• LLL gives a non-constructive proof that
Åi Eic ;.
• Can we efficiently find a point in that set?
• Long history:Beck ‘91, Alon ‘91, Molloy-Reed ‘98, Czumaj-Scheideler ‘00, Srinivasan ‘08
• Can find a satisfying assignment for k-SAT instanceswhere each variable appears in · 2k/7 clauses.
Algorithmic Results• Efficiently finding a point in Åi Ei
c.
• Moser-Tardos ‘10: Algorithm for General LLL in “variable model”.– Probability space has product distribution
on variables.– Each events is a function of some subset
of the variables.– Dependency graph connects events that
share variables.
• Many extensions: (but still in variable model)
– Cluster Expansion criterion [Pegden ‘13]– Shearer’s criterion [Kolipaka-Szegedy
‘11]
Algorithmic Results• Efficiently finding a point in Åi Ei
c.
• Moser-Tardos ‘10: Algorithm for General LLL in “variable model”.– Probability space has product distribution
on variables.– Each events is a function of some subset
of the variables.– Dependency graph connects events that
share variables.
• Many extensions in other directions– Derandomization [Chandrasekaran et al.
‘10]– Exponentially many events [Haeupler,
Saha, Srinivasan ‘10]– Column-sparse IPs [Harris, Srinivasan
‘13]– Distributed algorithms [Chung, Pettie, Su
‘14]
Extending LLL Beyond Dependency Graphs
• The definition of a dependency graph is equivalent to
P[Ei | Åj2J Ejc ] = P[Ei] 8J µ [n]n¡+(i).
• Define a lopsidependency graph to be one satisfying
P[Ei | Åj2J Ejc ] · P[Ei] 8J µ [n]n¡+(i).
• Intuitively, edges between events that are “negatively dependent”
• (Easy) Theorem: (Erdos-Spencer ‘91) All existential forms of the LLL remain true with lopsidependency graphs.
J = any set of non-neighbors
Extending LLL Beyond Dependency Graphs
• The definition of a dependency graph is equivalent to
P[Ei | Åj2J Ejc ] = P[Ei] 8J µ [n]n¡+(i).
• Define a lopsidependency graph to be one satisfying
P[Ei | Åj2J Ejc ] · P[Ei] 8J µ [n]n¡+(i).
• Intuitively, edges between events that are “negatively dependent”
• Eg: Let ¼ be a permutation on {1,2}.Let E1 be “¼(1)=1” and E2 be “¼(2)=2”.Then P[E1 | E2
c ] = 0 < 1/2 = P[E1 ].
J = any set of non-neighbors
E1 and E2 are “positively
dependent”
Extending LLL Beyond Dependency Graphs
• The definition of a dependency graph is equivalent to
P[Ei | Åj2J Ejc ] = P[Ei] 8J µ [n]n¡+(i).
• Define a lopsidependency graph to be one satisfying
P[Ei | Åj2J Ejc ] · P[Ei] 8J µ [n]n¡+(i).
• Intuitively, edges between events that are “negatively dependent”
• Eg: Let ¼ be permutation on [n]. [Erdos-Spencer ‘91, Harris-Srinivasan ‘14]Let Ei be “¼(ai)=bi”.Add edges between Ei and Ej if ai=aj or bi=bj.
J = any set of non-neighbors
• Define a lopsidependency graph to be one satisfying
P[Ei | Åj2J Ejc ] · P[Ei] 8J µ [n]n¡+(i).
• Definition: A lopsided association graph is a graph where
P[Ei |F ] ¸ P[Ei] 8F2Fi,where Fi contains all monotone functions of { Ei :
j¡+(i) }.
• Easy: Lopsided association ) Lopsidependency.
• Analogous to “negative association” vs “negative cylinder dependent”
Lopsided Association
• Define a lopsidependency graph to be one satisfying
P[Ei | Åj2J Ejc ] · P[Ei] 8J µ [n]n¡+(i).
• Definition: A lopsided association graph is a graph where
P[Ei |F ] ¸ P[Ei] 8F2Fi,where Fi contains all monotone functions of { Ei :
j¡+(i) }.
• Easy: Lopsided association ) Lopsidependency.
• Main Result: For every probability space with a lopsided association graph, all existential forms of LLL have an algorithmic proof.
Lopsided Association
Algorithmic LLL
Sym
metr
ic
Gen
era
l
Clu
ster
Exp
an
sion
Sheare
rVariable model
Dependency graphLopsided
association graphLopsidepend
ency graph Mos
er-Ta
rdos
Pegd
enKo
lipak
a-Sz
eged
y
Other Distributions
Sym
metr
ic
Gen
era
l
Clu
ster
Exp
an
sion
Sheare
rVariable model
Dependency graphLopsided
association graphLopsidepend
ency graph Mos
er-Ta
rdos
Pegd
enKo
lipak
a-Sz
eged
y
Permutations
Harris
-Srin
ivas
an
Algorithmic LLL beyond variable model
Sym
metr
ic
Gen
era
l
Clu
ster
Exp
an
sion
Sheare
rVariable model
Perfect Matchings in K n
Dependency graphLopsided
association graphLopsidepend
ency graph Mos
er-Ta
rdos
Pegd
enKo
lipak
a-Sz
eged
y
PermutationsAc
hlio
ptas
-
Iliop
oulo
s
Spanning Trees in K n?
Algorithmic LLL beyond variable model
* Needs slack+ Hamilton cycles in Kn...
Sym
metr
ic
Gen
era
l
Clu
ster
Exp
an
sion
Sheare
rVariable model
Perfect Matchings in K n
Dependency graphLopsided
association graphLopsidepend
ency graph Mos
er-Ta
rdos
Pegd
enKo
lipak
a-Sz
eged
y
Permutations
Achl
iopt
as-
Iliop
oulo
s
Spanning Trees in K n? * Needs
slack+ Hamilton. Cycles...
Algorithmic LLL beyond variable model
Algorithmic LLL beyond variable model
Sym
metr
ic
Gen
era
l
Clu
ster
Exp
an
sion
Sheare
rVariable model
Perfect Matchings in K n
Dependency graphLopsided
association graphLopsidepend
ency graph
Permutations
Spanning Trees in K n
All probability spaces
Our Results
* Need slackin Shearer’scriterion
* Intractability?
Resampling Oracles• Need some algorithmic access to probability
space ¹• Def: A resampling oracle for Ei is ri : ! satisfying– (R1) If !»¹|Ei then ri(!)»¹
Removes conditioning on Ei
– (R2) If j¡+(i) and Ej does not occur in !, then Ej does not occur in ri(!).
Cannot cause non-neighbors to occur
• Roughly, ri implements a coupling between ¹|Ei and ¹such that (R2) holds.
Resampling Oracles• Need some algorithmic access to
probability space ¹• Def: A resampling oracle for Ei is ri :
! satisfying– (R1) If !»¹|Ei then ri(!)»¹
– (R2) If j¡+(i) and Ej does not occur in !, then Ej does not occur in ri(!).
• Eg: Moser-Tardos Variable modelE1
Clause 1 unsatisfiedE2
Clause 2 unsatisfied
E3
Clause 3 unsatisfiedE4
Clause 4 unsatisfied
Á = (x1Çx2Çx3) Æ (x1Çx4Çx5) Æ (x4Çx5Çx6) Æ
(x2Çx3Çx6)ri(!): resample all variables in clause i.
Resampling Oracles• Need some algorithmic access to probability space
¹• Def: A resampling oracle for Ei is ri : ! satisfying– (R1) If !»¹|Ei then ri(!)»¹
– (R2) If j¡+(i) and Ej does not occur in !, then Ej does not occur in ri(!).
• Theorem: For any probability space and any events, TFAE– Existence of a lopsided association graph– Existence of resampling oracles.
• No guarantees it is compactly representable or efficient.What if events involve some NP-hard problem?
Resampling Oracles• Def: A resampling oracle for Ei is ri : !
satisfying– (R1) If !»¹|Ei then ri(!)»¹
– (R2) If j¡+(i) and Ej does not occur in !, then Ej does not occur in ri(!).
• Theorem: For any probability space and any events, TFAE– Existence of a lopsided association graph– Existence of resampling oracles.
• No guarantees it is compactly representable or efficient.What if events involve some NP-hard problem?
Spanning Trees in Kn
• Let T be a uniformly random spanning tree in Kn
• For edge set A, let EA = { AµT }.
• Lopsidependency Graph: Make EA
a neighbor of EB, unless A and B are vertex-disjoint.
• Resampling oracle rA:
– If T uniform conditioned on EA, want rA(T ) uniform.
–But, should not disturb edges that are vtx-disjoint from A.
Dependency
• EA = { AµT }.
• Resampling oracle rA(T ):
– If T uniform conditioned on EA, want rA(T ) uniform.
–But, should not disturb edges that are vtx-disjoint from A.A
T
–Contract edges of Tvtx-disjoint from A
–Delete remaining edgesvtx-disjoint from A
–Let rA(T ) be a uniformly random spanning tree in resulting graph.
rA(T )
Main Theorem
• Given–any probability space and events–a lopsided association graph– resampling oracles for the events
• Suppose that General LLL, Cluster Expansion or Shearer* holds.
• There is an algorithm to find a point in Åi Ei
c,running in oracle-polynomialy time, whp.
* Need slack for Shearer’s condition.y Actually depending on parameters in General LLL, etc.
Main Theorem• Given–any probability space and events–a lopsided association graph– resampling oracles for the events
• General LLL: 9 x1,...,xn 2 (0,1) such thatP[Ei] · xi ¢ j2¡(i) (1-xj) 8i.
• There is an algorithm to find a point in Åi
Eic,
using oracle calls, whp.
Main Theorem• Given–any probability space and events–a lopsided association graph– resampling oracles for the events
• Cluster Expansion: 9 y1,...,yn > 0 such that
P[Ei] · yi / Jµ¡+(i) yJ 8i.
• There is an algorithm to find a point in Åi Eic,
using oracle calls, whp.
J 2 Ind
The AlgorithmMaximalSetResample
Draw ! from ¹Repeat
J Ã ;While there is i¡+(J) s.t. Ei occurs
in !Pick smallest such iJ Ã J [ {i}! Ã ri(!)
EndUntil J = ;
• Similar to parallel form of Moser-Tardos
AnalysisReview of Moser-Tardos
E3
E9
E2
E5
E1
E7
E1
E3
E9
…
“Log” or “History” of resampling operations
Start
Dependency graph edges
AnalysisReview of Moser-Tardos
E3
E9
E2
E5
E1
E7
E1
E3
E9
…
“Log” or “History” of resampling operations
Start
Dependency graph edges
Witness tree
MaximalSetResampleDraw ! from ¹Repeat
J Ã ;While there is i¡+(J) s.t. Ei occurs
in !Pick smallest such iJ Ã J [ {i}! Ã ri(!)
EndUntil J = ;
J1, J2, J3, …Resampling sequence:
Ji+1 µ ¡+(Ji)Ji 2 Ind,
• Trivially, E[ # iterations ] = § Pr[ I1,I2,… is a prefix of resampling sequence ]
• Claim: Pr[ I1,…,Ik is a prefix ] · 1·i·k pIi
• Like Moser-Tardos, this is a coupling argument.But instead of using fact that variable have “fresh samples” whenever examined, we use the fact that ri returns thestate to the underlying distribution ¹.
I1,I2,…
• Trivially, E[ # iterations ] = § Pr[ I1,I2,… is a prefix of resampling sequence ]
• Claim: Pr[ I1,…,Ik is a prefix ] · 1·i·k pIi
• Proof: Like Moser-Tardos, using ri returns state to ¹.
• Claim: Let P[Ei] · (1-²) xi ¢ j2¡(i) (1-xj) (General LLL)Then § 1·i·k pIi ·
• Proof: Key idea: inductively show that § 1·i·k pIi ·
I1,I2,…
I1,…,Ik
I1,…,IkI1 = J
Comparison to Fotis’ work• Pros for Fotis:– Don’t need to sample from ¹ (there is no
¹)– Can handle scenarios that are not even
a lopsidependency graph,e.g., Hamilton cycles, Vtx Coloring.
• Pros for us:– Characterize when resampling
operations exist– Gives new proof of General LLL with
Dependency Graphs,in full generality. (Even for Shearer & Lopsided Association Graphs).
– Don’t need any slack in General LLL or Cluster Expansion conditions
– “Fractional transitions” seem easier in our setting