Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with...
-
Upload
natalie-stevenson -
Category
Documents
-
view
215 -
download
0
Transcript of Institute for Advanced Study, April 16 2012 Sushant Sachdeva Princeton University Joint work with...
Institute for Advanced Study, April 16 2012
Sushant SachdevaPrinceton University
Joint work with Lorenzo Orecchia, Nisheeth K. Vishnoi
Linear Time Graph Partitioningvia Fast Simulation of Random walks
INTRODUCTIONTheory of near-linear time algorithms
Why linear-time algorithms?
Input graphs and data sets are massive.e.g. Web graph (1011 nodes), Social networks (109 nodes)
Super-linear algorithms are impractical.
© 2004–2012 Michael K. Bergman.
Need time algorithms for fundamental problems:e.g. MAX-MATCHING, s-t MINCUT/MAXFLOW, Balanced Graph Partitioning
Approximate solutions maybe?
(1-²) approx., time[Duan and Pettie]
(1-²) approx., time[Christiano et al.]
Spectral approx., time
[This talk]
GRAPH PARTITIONING
Conductance
Given a undirected, unweighted graph
S Conductance of
[Cheeger, Alon-Milman]
[Spielman-Teng] allows us to approximate ¸2(L) and find such a cut in
time
Assume G is d-regularSpectral approximation
Conductance of G
Fundamental quantity in Markov Chains, Riemannian
Manifolds
Balanced SeparatorGiven b, does G have a b-balanced cut of
conductance < °?S
G NP-Hard
Applications to clustering, image segmentation, community detection,
primitive for divide-and-conquerAlgorithm Techniqu
eDistinguishes ≤ ° and
Running Time
Recursive Eigenvector
Spectral
Use Cheeger to find a cut and
remove it. Rinse and repeat.
Can take time. Too slow.
Balanced SeparatorGiven b, does G have a b-balanced cut of
conductance < °?S
G NP-Hard
Algorithm Technique
Distinguishes ≤ ° and
Running Time
Recursive Eigenvector
Spectral
[Leighton-Rao] Flow [AK,OSVV,CKM+]
[Arora-Rao-Vazirani]
SDP [AK,She,CKM+]
[Madry]
Balanced SeparatorGiven b, does G have a b-balanced cut of
conductance < °?S
G NP-Hard
Algorithm Technique Distinguishes ≤ ° and
Running Time
Recursive Eigenvector
Spectral
[Spielman-Teng] Local Random walks
[Andersen-Peres] Evolving Sets
[Orecchia-Vishnoi] SDP
Balanced SeparatorGiven b, does G have a b-balanced cut of
conductance < °?S
G NP-Hard
Algorithm Technique Distinguishes ≤ ° and
Running Time
Recursive Eigenvector
Spectral
[Spielman-Teng] Local Random walks
[Andersen-Peres] Evolving Sets
[Orecchia-Vishnoi] SDP
[This talk] Simulating Random walks
Balanced SeparatorGiven b, does G have a b-balanced cut of
conductance < °?S
G NP-Hard
Algorithm Technique
Distinguish ≤ ° and
Running Time
Recursive Eigenvector
Spectral
[Leighton-Rao] Flow [AK,OSVV,CKM+]
[Arora-Rao-Vazirani]
SDP [AK,She,CKM+]
[Madry]
[This talk] Simulating random walks
Overview
1. Use continuous-time random walks to give an algorithm
2. Show how to simulate continuous-time random walks fast
Graph Cuts and Random Walks
Intimate connection between Graph cuts and Random walks.
[Alon-Milman][Mihail][Lovasz-Simonovits][Spielman-Teng][Anderson-Chung-Lang][Andersen-Peres][Orecchia-Vishnoi][this talk]…
[Mihail] Given the distribution of a random walk that has mixed, can find a cut S with
in time.
Usual Random Walks on a graph
Page Rank Random walksEvolving Sets Random WalksContinuous Time Random Walks
Bird’s eye view1. Simulate random walks (one or several)
3. Else, modify the graph
Regular walk / Page Rank walk /Evolving Sets /Continuous-time walks
2. Try to cut the graph using the random walksIf there’s no low conductance cut, we get a certificate.If you find a balanced low conductance cut, we’re done. To remove the unbalanced low conductance cuts
Remove the set
Add edges across the cut
Using matchings /flows /stars
Threshold cuts/SDP rounding
Continuous Time Random Walks
Thm. We can reduce Balanced Separator to simulating
heat-kernel random walks with
Given initial distribution ,
Random walk of length l ~ Poisson(t)
Heat-Kernel random walk
Why heat-kernel random walk?
e-x is a nice function. Track progress using Golden Thompson inequality [AK]Don’t know such a result for other random walks
Techniques from SDPs[OV], exponential updates aka Matrix Mult. Weights [AK]
Previous results had a poly(1/°) dependence
Algorithm
Wej
An embedding in Random walk from vertex i
Case 1: All vectors are small
Long vectors are random walks that
haven’t mixed
Wei
All random walks have mixed well.
No small cuts.
Ensures mixing across cuts with
conductance more than °
Algorithm
Case 2: Walks have not mixed
Wej
WeiRandom Projection
+Threshold Cut
“Cheeger Rounding”
If we find a balanced cut of small conductance , we’re
done.ELSE
Algorithm
Case 2b: Walks have not mixed (and we didn’t find a balanced cut)
Wej
WeiWe can find a ball cut S
S
S is the union of “all unbalanced low
conductance cuts”
Algorithm
Case 2b: Walks have not mixed (and we didn’t find a balanced cut)
Wej
WeiWe can find a ball cut S
SAdd edges from all vertices outside the ball to all inside
Soft removal of unbalanced cuts
O(log n) rounds
For efficiency, we use JL lemma and work with
O(log n) dimensional embeddings. [AK]Goal: Given L,¿,v. Compute Goal: Given L,¿ ,v. Approximate
SIMULATING RANDOM WALKS
Approximating Heat-Kernel
Goal: Given ¿, L, v, ±, find u s.t.
Taylor Series Approx.?
Need
Too Slow
Heat-Kernel random walk required by
algorithm
Small error suffices for the algorithm Used in
previous algorithms
Approximating Heat-Kernel
Better polynomials?
YES!!
[Thm] There exists a polynomial p of degree such that
It suffices to approximate e-x on
[Thm] Given G that has a balanced cut of conductance , we can find one of conductance
in time Better guarantee, same
running time as [Andersen-Peres]
Still not near-linear!
Goal: Given ¿, L, v, ±, find u s.t.
Approximating Heat-Kernel
[Thm] Any polynomial such that requires degree
Even better polynomials?
Not Really!
Simple proof using Markov’s theorem
Goal: Given ¿, L, v, ±, find u s.t.
Beyond PolynomialsRational Functions![Saff-Schönhage-Varga] There is a degree k polynomial pk such that
Infinite Interval!
Geometric Decay!Need
degree
Assume: 1. We knew pk explicitly 2. ST computation was exact
We’re DONE!
Two Issues: 1. [SSV] result is existential
2. ST computation is approximate
Approximating up to error ±
Computing vectors
Can use [Spielman-Teng]![Thm] Given ¿, L, v, ±, we can compute u such
that in
time
[Spielman-Teng] (Informal) We can approximate (cI+L)-1y in
time.
LANCZOS’ METHOD“But we don’t know the polynomial.”
Lanczos’ MethodLet
Define f so that
Goal: Approximate f(B)v as well as any degree k poly.
Lanczos Method
Existence of a good approximate poly suffices
Small degree implies efficiency
From real approximation to matrices
Lanczos’ Method
Goal: Approximate f(B)v as well as any degree k poly.
Observe
p – any degree k polynomial
Krylov Subspace · of
order k
Let be an orthonormal basis for ·
Let T be the operator B restricted to · (k+1) x (k+1) matrix
For any degree k polynomial p,
Let Projection on · Our guess:
T is a much smaller matrix!
Error Guarantee
Goal: Bound the norm of the error
Let the error be
Equal!
Since T is B restricted to ·,Spectrum(T) is bounded by Spectrum(B)
* error of best degree k polynomial approximation
Cost = k mult. by B + Construct basis + Diagonalize T
for our setting
Suppose p(x) is a good degree k approximation to f
Bounded by r on Spectrum(B)
Bounded by r on Spectrum(T)
Don’t even need to know the polynomial!
HANDLING APPROXIMATE COMPUTATION
What about the error?
We bound the error
Why Lanczos is insufficient
Multiplication withis only approximate
Subspace · is approximate.& Operator T is
approximate
T is not even symmetric!
We lose nice spectral
properties
[ST] computation is only approximate
We symmetrize.Compute approximation with
We needed to compute an orthonormal basis for
Sketch of Error AnalysisBound
Bound BoundBound
Polynomial part
Error part
Rational approx. result
BoundBound sum of coefficients of
pk
Bound spectrum
of
Error Analysis + Lanczos method = Fast random walks
Fast random walks + Algorithm = Fast Balanced Separator
Conclusions
• Reduced balanced separator to simulating heat-kernel random walks
• Approximated the heat kernel random walk in time.
• Can be used as primitives for designing near-linear time algorithms.
Open Questions
• Other applications of fast matrix exponentiation
• Linear-time Graph decomposition?• Linear time algorithms for small set
expansion?• Linear time algorithm for
approximation for Balanced Separator
Thank you!
An Important Tool[Christiano et al.]
s-t MAXFLOW s-t MINCUT
Electrical Flows
Solve Linear Systems
Lx=y
L is a graph Laplacian
Solving Laplacian Linear Systems
Direct Methods
Gaussian Elimination /
Cholesky Decomposition
Iterative methods
Too slow. O(n3) time
Hopeless
Conjugate Gradient
Approximate method.
time
[Spielman-Teng] (Informal) We can approximate L-1y in time.
Other Applications – Approximating second eigenvector [Spielman-Teng], Cover times [Ding-Lee-
Peres], Generating random spanning trees[Madry-Kelner], Graph Sparsification [Spielman-Srivastava]
Balanced graph partitioning,Simulating continuous-time random walks
[This talk]
Beyond Polynomials
Rational Functions!
[Saff-Schönhage-Varga] There is a degree k polynomial pk such that
Infinite Interval!
Geometric Decay!Small required
degree.
L1 bound on [0,1)
L1 bound on [-1,1]
L1 bound on [-1,1]
L2bound on [-1,1]
Change IntervalWrite function as an
integralCauchy Schwarz
Assume: 1. We knew pk explicitly 2. ST Solver was exact
We’re DONE!
Two Issues: 1. We don’t know pk explicitly 2. ST Solver
is approximate
Faster SimulationPolynomials don’t suffice, then, how do we simulate the random walk fast?
Lanczos Method
Goal: Given symmetric B, v, k, and function f;approximate f(B)v as well as any degree k poly
Observe
p – any degree k polynomial
Krylov Subspace · of
order kLet be an orthonormal basis for ·
k-Identity Projection on
·Let
For any degree k polynomial p,
Our guess:
T is a much smaller matrix!
Error Guarantee
Goal: Bound the norm of the error
Write Any degree
k polynomial
Equal!
Since and the columns of V are orthnormal,
The columns of V are orthnormal
True for any p!!
As good as the best degree k polynomial!
Cost = k mult. by B + Construct basis + Diagonalize T
if B is a Laplacian
Using Lanczos for our Algorithm
Goal: Given A, wanted to approximate as
Use Lanczos with
suffices.
Still need to address the issue of approximate multiplication with