Cuts, Trees, and Electrical Flows

Aleksander Mądry

Graphs are everywhere!

(In case you just got here by mistake…)

They got HUGE

(Could allow directed edges, but will focus on undirected today) Algorithmic Graph Theory:Developing algorithmic ideas and tools to allow us

to analyze and understand graphs

Example Tasks of Interest

Graph Partitioning (clustering, community detection, div & conq)

Connectivity Analysis(congestion estimation,

analyzing resiliance to link failures)

Network Design(supporting most efficentcommun. infrastructure,

routing schemes)

(Could allow directed edges, but will focus on undirected today) Algorithmic Graph Theory:Developing algorithmic ideas and tools to allow us

to analyze and understand graphs

Big Data = Big ProblemWe need our algorithms to be really fast

(theory paradise: asymptotic complexity does matter here,O(n2) or even O(n3/2) does not cut it)

Also, parallelization/distributed aspects play role

Classic approaches that we used so far are often too slow and hard to parallelize/decentralize

Fundamental question: What graph problems can we solve

in nearly-linear time?

Undirected graph G withinteger capacities u( )∙

The setup8

11

4

19

103

720

1

5

68

6

15

We want to solve minimization cut-based problems on G

Cut-basedmin problem

C

u(C) = capacity of the cut C f(C) = arbitrary positive function independent of edges/capacities

f(C) = 1 → minimum cut problemf(C) = 1 if C separates s and t; f(C)=∞ otherwise

→ minimum s-t cut problemf(C) = 1/min {|C|, |V\C|} → sparsest cut problem

Examples:

• generalized sparsest cut problem• minimum conductance cut problem• balanced separator problem• minimum bisection problem• minimum multi-cut problem• minimum multiway-cut problem• …

Other important examples:

Can we solve these problems in poly time?

Yes, in case of minimum (s-t) cut,but the remaining ones are NP-hard

Good news: Can get a O(log n) (or better) approx.!

What about nearly-linear time computations?

Bad news: Known alg. for min s-t cut need Ω(n3/2) timeSame is true for (almost) all the best approx. algorithms

for the other problems

How about trading approx. for efficiency?

(We already did this to cope with NP-hardness)

Good news: Possible for minimum cut problem! [Karger ’00]

Lots of progress in the context of sparsest cut and minimum conductance cut

[Arora Hazan Kale ‘04 Khandekar Rao Vazirani ’06 Arora Kale ‘07 Orecchia Schulman Vazirani Vishnoi ‘08 Sherman ‘09 Spielman Teng ’04 Andersen Chung Lang ’06

Andersen Peres ‘09 Orecchia Vishnoi ’11 Orecchia Sachdeva Vishnoi ‘12]

Still-unanswered question: Can we design nearly-lineartime approx. algorithms for all these problems?

[M. ’10]: Yes, if we can compute such a fast approx. on treesMore precisely: Fix any cut-based minimization problem

Good and fast β-approx. alg. working only on trees↓

A O(β polylog)-approx. for general graphs that runs in Õ(n1+δ) time

(The poly-log grows mildly with 1/δ)

Moral: When shooting for a fast polylog-approx. algorithm for a cut-based minimization problem,

just focus on trees

G

CC

C

λ1

λi

(Ti ,λi)

(T1 ,λ1)

How to prove such a theorem?

such that for any cut C:

(cut lower-bounding) ui(C) ≥ u(C) for all i

(cut upper-bounding) Eλ[u(C)]:=∑i λi ui(C) ≤ O(log n) u(C)

[Räcke ‘08] (simplified): For any graph G=(V,E,u), we can finda convex combination {(λi,Ti)}i of trees*

How to use this decomposition?

Idea for lifting: 1) Find {(λi,Ti)}i as described2) Sample a tree T being Ti with prob. λi

3) Output a β-optimal solution C for instance P on T

≤ β uT(C*)fP(C*) ≤ O(β log n) u(C*) fP(C*)=O(β log n) OPT

Why should it work?

With prob. ≥ 1/2: uT(C*)≤ O(log n) u(C*)Let C* be the optimal solution

But u(C) fP(C) ≤ uT(C) fP(C)

Note: Choice of T is oblivious to the problem we want to solve

Lifting works great! How about running time? Räcke’s algorithm runs in Õ(n3) time Prohibitive from our point of view! What to do now?

Idea: Decompose G into objects that are more complicated than trees,

but still simpler than general graphs

H is a j-tree if it is a union of:→ a forest F (envelope)→ an arbitrary graph R (core)

and:1) |V(R)|≤ j 2) for each connected component F’ of F, |V(F’)∩V(R)|=1

Note: 1-tree is just a tree

HRF

Decomposing graphs into j-trees

The real benefit comes from the ability to vary j!

Theorem (simplified): For any graph G=(V,E,u) and j≥1, we can find in Õ(m+n2/j) time a convex comb. {(λi,Ti)}i of j-trees s.t. for any cut C:

(cut lower-bounding) ui(C) ≥ u(C) for all i (cut upper-bounding) Eλ[u(C)]:=∑i λi ui(C) ≤ Õ(log n) u(C)

j=1 yields Räcke’s result with faster running time but has slightly worse quality

G

Speeding up min s-t cut computation:

Õ(m+n2/j) time+ sampling

C*

TR

C*

With prob. 1/2, cut C* isÕ(log n)-preserved

If we run the min s-t cut alg. on T instead of G then appropriate choice of j gives a speed up!

(i.e. if j is such that we minimize the total running time: Õ(m+n2/j) + Õ(m+j(1+c)) << Õ(m+n1+c))

s

t

s

t

G

Õ(m+n2/j) time+ sampling

C*

TR

C*

s

t

s

t

We get a running time arbitrarily close to nearly-linear

There is even better way of leveraging this flexibility!

…we do it in a series of small recursive steps

…but at a price of approx. ratio growing accordingly

Instead of reducing G to T in one big step…

Success?We get a good generic „baseline” solution

for a large family of problems

But…The approx. ratio could be better

Algorithms „seem” not really practical yet(Can we use some of the insight to speed up

the existing heuristics?)

Still…The approx ratio should be improvable

Was recently used to get (1+ε)-approx. to undirected max flow in close-to-linear time

[Kelner Lee Orecchia Sidford ‘13] [Sherman ‘13]

Spectral Graph TheoryBeyond λ2

=L

Spectral graph theory:Connecting combinatorial structure of G

to linear-algebraic properties of L

The Laplacian matrix of a graph G

AD -

D = diagonal vertex degree matrixA = adjacency matrix

Key quantity: λ2 = second-smallest eigenvalue of L

Classic success story: λ2 connection

Lots of connections to spectral properties of the graph(e.g., understanding of random walks)

Spectral (and local) partitioning:Works great on well-connected graphs

(i.e., graphs with large λ2)

But has well-known limitations

Can we take spectral graph theory ‘beyond’ λ2 ?

Promising example:O(k2 λ2 (λk)-1/2)-approx. to uniform sparsest cut

[Kwok Lau Lee Oveis-Gharan Trevisan ’13](If λk fairly large for some small k a very good approx.)

Upshot: λk fairly large G = union of k-1 expander-like graphs

Still: Can we access the whole spectrum of Lin more principled manner?

Given a graph G with resistances {re}e

source s and sink t

s t

resistances given by {re}e

Recipe for elec. flow:1) Treat edges as resistors 2) Connect a battery to s and t

Object of interest:Electrical flows

Given a graph G with resistances {re}e

source s and sink t

s t

Recipe for elec. flow:1) Treat edges as resistors 2) Connect a battery to s and t

induced current(flow conservation

constraints hold)

Object of interest:Electrical flows

Computing an electrical flow= solving a Laplacian linear system

=L x b

What does it have to do with Laplacians?

Furthermore: [Spielman Teng ’04, Koutis Miller Peng ’10 ‘11] We can (essentially) solve such systems in nearly-linear time

Result: Electrical flow is a nearly-linear time primitive

Fast maximum/multi-commodity flow approximation[Christano Kelner M. Spielman Teng ‘11] [Kelner Miller Peng ‘12]

How can we employ this primitive?

Faster random walk simulations[Kelner M. ‘09]

Where else will electrical flow be a useful primitive?

Conclusion

Power of tree-like decompositions: Approx. vs. running time trade-off for

undirected cut-based minimization problems

Outstanding challenge: Directed graphs

Spectral graph theory beyond λ2: Electrical flows primitive

Thank you

Questions?