Geometry and Expansion: A survey of some results Sanjeev Arora Princeton ( touches upon: S. A.,...

Geometry and Expansion: A survey of some results

Sanjeev Arora Princeton

( touches upon:

S. A., Satish Rao, Umesh Vazirani, STOC’04; S. A., Elad Hazan, and Satyen Kale, FOCS’04; S. A., James Lee, and Assaf Naor, STOC’05 & JAMS’08 S.A., S. Kale STOC 2007.

+ papers that are not mine)

Outline:

• Graph partitioning problems: intro and history

•New approximation via expander flows.

• New approximation algorithm via semidefinite programming (+ analysis using “Structure Theorem”) [A., Rao, Vazirani]

• Outline of proof of “S. T.”

• Uses of “S. T.” in geometric embeddings

• Open problems

Sparsest Cut / Edge Expansion

SS

G = (V, E)

c- balanced separator

Both NP-hard

G) = minS µ V

| E(S, Sc)|

|S||S| < |V|/2

c(G) = minS µ V

| E(S, Sc)|

|S|c |V| < |S| < |V|/2

Why these problems are important

• Analysis of random walks, PRAM simulation, packet routing, clustering, VLSI layout etc.

• Underlie many divide-and-conquer graph algorithms (surveyed by Shmoys’95)

• Discrete analog of isoperimetry; useful in Riemannian geometry (via 2nd eigenvalue of Laplacian (Cheeger’70)

• Graph-theoretic parameters of inherent interest (cf. Lipton-Tarjan planar separator theorem)

Previous approximation algorithms

1) Eigenvalue approaches (Cheeger’70, Alon’85, Alon-Milman’85)

Only yield factor n approximation. 2c(G) ¸ (G) ¸ c(G)2 /2

2) O(log n) -approximation via LP (multicommodity flows) (Leighton-Rao’88) • Approximate max-flow mincut theorems

• Region-growing argument

(Linial, London, Rabinovich’94, AR’94)

3) Embeddings of finite metric spaces into l1

• Geometric approach; more general result (but still O(log n) approximation)

New results of [ARV’04]

1. O( ) -approximation to sparsest cut and conductance

2. O( )-pseudoapproximation to c-balanced separator (algorithm outputs a c’-balanced separator, c’ < c)

3. Existence of expander flows in every graph (approximate certificates of expansion)

log n

log n

Disparate approaches from previous slide get “unified”

Subsequent work: [AHK’05],[AK’07],[S’09]: O(m + n1.5 ) time!

The three main characters

Expansion Isoperimetry

(continuous analog of expansion)

Geometry

(and geometric embeddings of finite metric spaces)

Identifying sparse cuts via “traffic flows”

Approach 1: traffic congestion identifies sparse cuts

[SM’87]: Stress a network bypassing traffic “flow” throughit. Look at congested edgesto identify sparse cuts

[LR88] O(log n) approximationto sparsest cut. Route 1 unit of traffic between every pair of nodes

[ARV’04] Traffic flow is like embedding a weighted graph.wij = amount of traffic from i to j

Solve a math program to find the “right” flow pattern([AHK’05] Do it in O(n2) time)

Expander traffic flows [ARV’04]

G = (V, E)

SS

A D-regular flow graph s.t.

8 S w(S, Sc) = ( D |S|)

Weighted Graph w satisfies (*) iff L(w) = (1) [Cheeger]

(*)

Our Thm: If G has expansion , then a D-regular expander flow exists in it where D=

(certifies expansion = (D) )

Formal statement : 9 0 >0 s.t. foll. LP is feasible for D = (G)

log n

fp ¸ 0 8 paths p in G

8i j p 2 Pij fp = D (degree)

Pij = paths whose endpoints are i, j

8S µ V i 2 S j 2 Sc p 2 Pij fp ¸ 0 D |S| (demand graph is

an expander)

8e 2 E p 3 e fp · 1 (capacity)

WHY IS THIS FEASIBLE???

Feasibility Criterion for LP on prev. slide (via Farkas’s Lemma)

Existence of such i, j proved in [ARV’04]. When fail tofind such i, j, we find a cut of small expansion

Overall approximation algorithm via flows

Try to solve above LP to find D-regular expander flow

If succeed, have verified that expansion is D/10.

If fail, then use [ARV04] ideas to find a cut of capacity

Note: Before finding this cut already had D/2-regular flow

Next: The SDP-based approach to Graph partitioning (ARV’04)

c-balanced separator

c(G) = minS µ V

| E(S, Sc)|

|S|c |V| < |S| < |V|/2

SS

Assign {+1, -1} to v1, v2, …, vn to minimize

(i, j) 2 E |vi –vj|2/4

Subject to i < j |vi –vj|2/4 ¸ c(1-c)n2

+1

-1

|vi –vj|2/4 =1

Semidefinite relaxation for

Find unit vectors in <n

|vi –vj|2 + |vj –vk|2 ¸ |vi –vk|2 8 i, j, k

Triangle inequality

“cut semimetric”

|vi –vj|2 =0

Unit l22 space

Unit vectors v1, v2,… vn 2 <d


Vi

Vk

Vj

non obtuse !

Example: Hypercube {-1, 1}k

|u – v|2 = i |ui – vi|2 = 2 i |ui – vi| = 2 |u – v|1

In fact, l2 and l1 are subcases of l22

Structure Theorem for l22 spaces [ARV’04]

Subsets S and T are -separated if for every vi 2 S, vj 2 T |vi –vj|2 ¸

¸

Thm: If i< j |vi –vj|2 = (n2) then 9 S, T of size (n) that are -separated for = ( 1 )

<d

log n

Main thm ) O( )-approximationlog n

v1, v2,…, vn 2 <d is optimum SDP soln; SDPopt = (i, j) 2 E |vi –vj|2

S, T : –separated sets of size (n)

Do BFS from S until you hit T. Take the level of the BFS tree with the fewest edges and output the cut (R, Rc) defined by this level

(i, j) 2 E |vi –vj|2 ¸ |E(R, Rc)| £

) |E(R, Rc)| · SDPopt /

· O( SDPopt) log n

S

d(S, i)

i

j

d(S, j)

Other new -approximation algorithms

• MIN-2-CNF deletion and several graph deletion problems. [Agarwal, Charikar, Makarychev, Makarychev’04]. Weighted version of S.T.

• MIN-LINEAR ARRANGEMENT [Charikar, Karloff, Rao’04]

• General SPARSEST CUT [A., Lee, Naor ’04]

• Min-ratio VERTEX SEPARATORS and Balanced VERTEX SEPARATORS [ Feige, Hajiaghayi, Lee, ’04]

log n

All use the Structure Theorem (+ other ideas)

Outline:




• Uses of “S. T.” in geometric embeddings

• Introduction to expander flows and O(n2) time algorithms

• Open problems

S

T (Algorithm to produce -separatedsets S, T, of size (n) )

Algorithm to produce two –separated sets

<d

u

Su

Tu

0.01

d

Easy: Su and Tu likely to have size (n)

If Su, Tu still have size (n), output them

Main difficulty: Show that whp only o(n) points get deleted

d

“Stretched pair”: vi, vj such that |vi –vj|2 · and | h vi –vj, u i | ¸ 0.01

Obs: Deleted pairs are stretched and they form a matching.

Delete any vi 2 Su, vj 2 Tu s.t. |vi –vj|2 < . (till no such pair remains)

Naïve analysis of random projection fails

<dv

u

<u, v> ??1

d

1

d

e-t

2/2

Stretched pair: |vi –vj|2 < ; |<vi –vj, u>| > 0.01d

standard deviations

E[# of stretched pairs] = n2 exp(-) À n

= O( 1 )

Vi

Ball (vi , )u

Vj

0.01

d

Proof by contradiction: Suppose matching of (n) size exists with probability (1)…

….stretched pairs are almost everywhere you look!

Idea: Put stretched pairs together; derive very improbable event

Walks in unit l22 space

Unit vectors v1, v2,… vn 2 <d


Vi

Vk

Vj

Angles are non obtuse

Taking r steps of length s

only takes you squared distance rs2

(i.e. distance r s)

ss

ss

Proof by contradiction (contd.)

ss

ss r steps of length s )

squared distance rs2 (distance r s)

Stretched pair: |vi –vj|2 < ; |<vi –vj, u> ¸ 0.01

Claim: 9walk on stretched edges

u

d

0.01

d

0.01

d

….

<vfinal –v0, u> ¸ r 0.01

d|vfinal –v0| · r

Projection = r

£ standard deviation

VERY UNLIKELY IF r large enough) Walk impossible (CONTRADICTION)

Why walk is possible: delicate argument; measure concentration

Outline:




• Geometric embeddings of metric spaces

• Open problems

Finite metric space (X, d)

x

y

d(x,y)

<k (with l2 norm)

f

distortion of f is minimum C>1 such that

d( x, y) · |f(x ) – f( y)|2 · C d( x, y) 8 x, y

Thm (Bourgain’85): For every n-point metric space, a map exists with distortion O(log n)

[LLR’94]: Efficient algorithm to find the map; Proof that O(log n) cannot be improved in general

Qs: Improve O(log n) for X = l22 (say) or l1 ?

f(x)

f(y)

Embeddings and Cuts (LLR’94, AR’94)

Recall: Cut semi-metric

1

0

Fact: Metric (X, d) embeds isometricallyin l1 iff it can be written as a positive combination of cut semimetrics

Embedding l22 into l1 gives a way to produce cuts from SDP solution

Status report of this area

l1 into l2log0.5 n[Enflo’69]

l22 into l1 1.16 [Zatloukal’04]

Superconstant[Khot, Vishnoi’04](logn)0.01 [Cheeger,Kleiner, ‘08]

l22 into l2log0.5 n[Enflo’69]

Best lowerbound Best upperbound

Exactly the integrality gap of SDP for general SPARSEST CUT[LLR’94, AR’94]

log n[Bourgain’85]

log0.75 n

[Chawla,Gupta,Racke ’04]

log0.5 n log log n[A., Lee, Naor’04]

Uses fourier techniques developed for PCPs!

Disproves Goemans-Linial

conjecture

Uses new metric differentiation techniques

Upperbounds:Frechet’s recipe to embed metric space (X, d) into Rk

Pick k suitable subsets A1, A2, …, Ak of X

Map x 2 X to (d(x, A1), d(x, A2), … , d(x, Ak))Aix

In recent embeddings, Ai’s are chosen using S.T.and “Measured descent” idea of [Krauthgamer, Lee, Naor, and Mendel’04]

Note: d(x, A1) – d(y, A1) · d(x, y)

Embedding lowerbounds (Khot-Vishnoi’05)

Explicit unit- l22 space (X, d) that requires distortion log log log n into l1

Main observation: Need good handle on cut structure of X

Use hypercube as building block !

Cut ´ Boolean Function

Number of cut edges = average sensitivity

(Fourier analysis a la KKL, Friedgut, Hastad, Bourgain etc. ) isoperimetric theorems)

OPEN PROBLEMS

• Better approximation factor than O( )? (log log n “lowerbound” assuming UGC )

• Better distortion bound for embedding l22 into l1?.) ( upperbound v/s lowerbound

• Combinatorial approximation algorithms for other problems ? (similar to one for SPARSEST CUT from [A., Hazan, Kale] )

• Other applications of expander flows? (Useful in some geometric results [Naor, Rabani, Sinclair’04])

• Ways to use spectral ideas a la [ABS’10] for SPARSEST CUT?

Example of expander flow

n-cycle

Take any 3-regular expander on n nodesPut a weight of 1/3n on each edge

Embed this into the n-cycle

Routing of edges does not exceed any capacity ) expansion =(1/n)

Other extensions of flow-based techniques

• Generalization to problems other than sparsest cut [A., Kale07] “Primal-dual approach to SDP.”

• Very fast algorithms for O(log n) approximation: O(n1.5 + m) time (Faster than [LR88] type algorithms!)

• Very simple algorithms; use only maxflow and eigenvalue computations [KRV06]

Looking forward to more progress…

Thanks !

New Result (A., Hazan, Kale;FOCS’04)

O(n2) time algorithm that given any graph G finds for some D >0

• a D-regular expander flow • a cut of expansion O( D )log n

Ingredients: Approximate eigenvalue computations; Approximate flow computations (Garg-Konemann; Fleischer) Random sampling (Benczur-Karger + some more)

Idea: Define a zero-sum game whose optimum solution is an expander flow; solve approximately using Freund-Schapire approximate solver.

)D) · (G) ·O(D )log n

Expander flows: LP view

LP feasible ) ¸ (D)LP feasible ) ¸ (D)

GG

· D· D

· 1· 1

Thm [ARV]: 9 0 s.t. the LP is

feasible with D = /√log n

Thm [ARV]: 9 0 s.t. the LP is

feasible with D = /√log n

Open problems (circa April’04)

• Better running time/combinatorial algorithm?

• Improve approximation ratio to O(1); better rounding??(our conjectures may be useful…)

• Extend result to other expansion-like problems (multicut, general sparsest cut; MIN-2CNF deletion)

• Resolve conjecture about embeddability of l22 into l1; of l1 into l2

• Any applications of expander flows?

O(n2) time; [A., Hazan, Kale]

log3/4 n distortion; [Chawla,Gupta, Racke]

Integrality gap is (log n) [Charikar]

Yes [Naor,Sinclair,Rabani]

Better embeddings of lp into lq [Lee]

Various new results

O(n2) time combinatorial algorithm for sparsest cut (does not use semidefinite programs)

[A., Hazan, Kale’04]

New results about embeddings: (i) lp into lq [J. Lee’04]

(ii) l22 and l1 into l2 [CGR’04]

(approx for general sparsest cut)

Clearer explanation of expander flows and their connection to embeddings [NRS’04]

A concrete conjecture (prove or refute)

G = (V, E); = (G)

For every distribution on n/3 –balanced cuts {zS} (i.e., S zS =1)

there exist (n) disjoint pairs (i1, j1), (i2, j2), ….. such that for each k,

• distance between ik, jk in G is O(1/ )

• ik, jk are across (1) fraction of cuts in {zS}

(i.e., S: i 2 S, j 2 Sc zS = (1) )

Conjecture ) existence of d-regular expander flows for d =

log n

log n

nlog

Example of l22 space: hypercube {-1, 1}k

|u – v|2 = i |ui – vi|2 = 2 i |ui – vi| = 2 |u – v|1

In fact, every l1 space is also l22

Conjecture (Goemans, Linial): Every l22 space is l1 up to distortion O(1)

LP Relaxations for c-balanced separator

Motivation: Every cut (S, Sc) defines a (semi) metric

1

1

1

0 0

Xij 2 {0,1}

i< j Xij ¸ c(1-c)n2

Xij + Xj k ¸ Xik

0 · Xij · 1

Semidefinite

There exist unit vectors v1, v2, …, vn 2 <n such that Xij = |vi - vj|2 /4

Min (i, j) 2 E Xij

Semidefinite relaxation (contd)

Min (i, j) 2 E |vi –vj|2/4

|vi|2 = 1


i < j |vi –vj|2 ¸ 4c(1-c)n2

Unit l22 space

Many other NP-hard problems have similar relaxations.

Algorithm to produce two –separated sets

<d

u

Su

Tu

0.01

d

Check if Su and Tu have size (n)

If any vi 2 Su and vj 2 Tu satisfy

|vi –vj|2 ·

repeat until no such vi, vj remain

delete them and

If Su, Tu still have size (n), output them

Main difficulty: Show that whp only o(n) points get deleted

d


Obs: Deleted pairs are stretched and they form a matching.

Next 10-12 min: Proof-sketch of Structure Thm

( algorithm to produce -separated S, T of size (n); = 1/ )nlog

S

T

“Matching is of size o(n) whp” : naive argument fails

d


O( 1 ) £ standard deviation

) PrU [ vi, vj get stretched] = exp( - 1 )

= exp( - )log n

E[# of stretched pairs] = O( n2 ) £ exp(- ) log n

Generating a contradiction: the walk on stretched pairs

u

Vi

Vj

0.01

d

0.01

d

r steps

0.01

d

r

|vfinal - vi| < r

| <vfinal – vi, u>| ¸ 0.01rd

= O( r ) x standard dev.

vfinal

Contradiction if r is large enough!

Measure concentration (P. Levy, Gromov etc.)

<d

A

A : measurable set with (A) ¸ 1/4

A : points with distance · to A

A

A) ¸ 1 – exp(-2 d)

Reason: Isoperimetric inequality for spheres

Expander flows

(approximate certificates of expansion)

Geometry and Expansion: A survey of some results Sanjeev Arora Princeton ( touches upon: S. A.,...

Documents

Transcript of Geometry and Expansion: A survey of some results Sanjeev Arora Princeton ( touches upon: S. A.,...