Dependent Randomized Rounding in Matroid Polytopes (& Related

Results)Chandra Chekuri Jan Vondrak Rico Zenklusen

Univ. of Illinois IBM Research MIT

Example: Congestion Minimization

s3

s2

s1

t1

t2

t3

Choose a path for each pair

Minimize max number of paths using any edge (congestion)

Special case: Edge-Disjoint Paths

G

Example: Congestion Minimization

s3

s2

s1

t1

t2

t3

Choose a path for each pair

Minimize max number of paths using any edge (congestion)

Special case: Edge-Disjoint Paths

[Raghavan-Thompson’87]•Solve mc-flow relaxation (LP)•Randomly pick a path according to fractional solution•Chernoff bounds to show approx ratio of O(log n/log log n)

0.10.3

0.2

0.7

0.25

0.5

0.65

0.15

0.15

G

Chernoff-Hoeffding Concentration Bounds

• X1, X2, ..., Xn independent {0,1} random variables

• E[Xi] = Pr[Xi = 1] = xi

• a1, a2, ..., an numbers in [0,1]

• μ = E[Σi ai Xi] = Σi ai xi

Theorem: • Pr[Σi ai Xi > (1+δ) μ] ≤ ( e δ / (1+δ) δ ) μ

• Pr[Σi ai Xi < (1 - δ) μ] ≤ exp(- μ δ2/2)

Example: Multipath Routing

s3

s2

s1

t1

t2

t3

Choose ki paths for pair (si, ti)(assume paths for pair disjoint)

Minimize max number of paths using any edge (congestion)

k2 = 1

k1 = 2

k3 = 2

G

Example: Multipath Routing

s3

s2

s1

t1

t2

t3

Choose ki paths for pair (si, ti)(assume paths for pair disjoint)

Minimize max number of paths using any edge (congestion)

[Srinivasan’99]•Solve mc-flow relaxation (LP)•Randomized pipage rounding• O(log n/log log n) approx via negative correlation

0.25

0.3

0.7

0.7

0.30.5

0.80.5

0.95

G

Dependent Randomized Rounding

Randomized rounding while maintaining some dependency/correlation between variables

Dependent Randomized Rounding

Randomized rounding while maintaining some dependency/correlation between variables

Several variants in literatureThis talk: dependent randomized rounding to satisfy a

matroid base constraint while retaining concentration bounds similar to independent rounding

Briefly, related work on matroid intersection and non-bipartite graph matchings

Crossing Spanning Trees and ATSP

Undir graph G=(V,E)

Cuts S1, S2, …, Sm

Find spanning tree T that minimizes max # of edges crossing a given cut[Bilo-Goyal-Ravi-Singh-’04]

[Fekete-Lubbecke-Meijer’04]

Crossing Spanning Trees and ATSP

Undir graph G=(V,E)

Cuts S1, S2, …, Sm

Find spanning tree T that minimizes max # of edges crossing a given cut

[Asadpour etal]

• Solve LP: x point in spanning tree polytope of G

• Dependent rounding via maximum entropy sampling

• O(log m/log log m) approx

• Also O(log n/log log n) for ATSP (several other ideas)

1

0.4 0.6

0.9

0.4

0.3

1

0.7

0.7

Tool: Negative Correlation

• X1, X2 two binary ({0,1}) random variables

• X1, X2 are negatively correlated if E[X1 X2] ≤ E[X1] E[X2]

• That is, Pr[X1 = 1 | X2 = 1] ≤ Pr[X1 = 1] and

Pr[X2 = 1 | X1 = 1] ≤ Pr[X2 = 1]

Tool: Negative Correlation

• X1, X2 two binary random variables

• X1, X2 are negatively correlated if E[X1 X2] ≤ E[X1] E[X2]

• That is, Pr[X1 = 1 | X2 = 1] ≤ Pr[X1 = 1] and

Pr[X2 = 1 | X1 = 1] ≤ Pr[X2 = 1]

• Also implies (1-X1), (1-X2) are negatively correlated

Negative Correlation

• X1, X2, ..., Xn binary random variables

• X1, X2, ..., Xn are negatively correlated if for any index set J {1,2, ..., n}• E[ i J Xi ] ≤ i J E[Xi ] and• E[ i J (1-Xi)] ≤ i J E[(1-Xi)]

Negative Correlation and Concentration

• X1, X2, ..., Xn binary random variables that are negatively correlated (can be dependent)

• E[Xi] = Pr[Xi = 1] = xi

• a1, a2, ..., an numbers in [0,1]

• μ = E[Σi ai Xi] = Σi ai xi

Theorem: [Panconesi-Srinivasan’ 97]• Pr[Σi ai Xi > (1+δ) μ] ≤ ( e δ / (1+δ) δ ) μ

• Pr[Σi ai Xi < (1 - δ) μ] ≤ exp(- μ δ2/2)

Connecting the dots ...

What is common between the two applications?

Integer Program:

min λs.tA x ≤ λbx is a base in a matroid

A non-neg matrix, packing constraints

Multipath: x corresponds to choosing ki paths for pair siti from Pi Crossing tree: x induces a

spanning tree

congestion

Matroids

M=(N, I ) where N is a finite ground set and I 2N is a set of independent sets such that

• I is not empty• I is downward closed: B I and A B A

I• A, B I and |A| < |B| implies there is i B\A

such that A+i I

Matroid Examples

• Uniform matroid: I = { S : |S| ≤ k }• Partition matroid: I = { S : |S Nj| ≤ kj, 1 ≤

i ≤ h } where N1, ..., Nh partition N, and kj are integers

• Graphic matroid: G = (V, E) is a graph and M=(E, I) where I = { S E : S induces a forest }

Bases in Matroid

• B I is a base of a matroid M=(N, I) if B is a maximal independent set

• All bases have same cardinality• Matroids can also be defined via bases• Example: spanning trees in a graph

Base Exchange Theorem

B’ and B’’ are distinct bases in a matroid M=(N, I)

Strong Base Exchange Theorem: There are elements i B’\B’’ and j B’’\B’ such that B’-i+j and B’’-j+i are both bases.

B’ B’’

B’B’’ B’B’’

i j B’-i+j and B’’-j+i are both bases

Dependent Rounding in Matroids

• M = (N, I ) is a matroid with |N| = n• B(M) is the base polytope: conv{1B : B is a

base}• x is a fractional point in B(M)• Round x to a random base B such that • Pr[i B] = xi for each i N• Xi (indicator for i B ) variables are negatively

correlated

Our Work

Two methods for arbitrary matroids:1. Randomized pipage rounding for matroids

[Calinescu-C-Pal-Vondrak’07,’09]2. Randomized swap rounding

[C-Vondrak-Zenklusen’09]This talk: randomized swap rounding

Randomized Swap Rounding

• Express x = mj=1 βi Bi (convex comb. of

bases)• C1 = B1 , β = β1

• For k = 1 to m-1 do • Randomly Merge β Ck & βk+1 Bk+1 into (β+βk+1)

Ck+1

• Output Cm

Swap Rounding

0.2 C1 + 0.1 B2 + 0.5 B3 + 0.15 B4 + 0.05 B5

0.3 C2 + 0.5 B3 + 0.15 B4 + 0.05 B5

0.8 C3 + 0.15 B4 + 0.05 B5

0.95 C4 + 0.05 B5

C5

x = 0.2 B1 + 0.1 B2 + 0.5 B3 + 0.15 B4 + 0.05 B5

0.90.31

0.4 0.6

0.41

0.7

0.7

0.3 0.6

0.1

Merging two Bases

Merge B’ and B’’ into a random B that looks like B’ with probability p and like B’’ with probability (1-p)

Merging two Bases

Merge B’ and B’’ into a random B that looks like B’ with probability p and like B’’ with probability (1-p)

Option: Pick B’ with prob. p and B’’ with prob. (1-p) ?

Will not have negative correlation properties!

Merging two Bases

B’ B’’

B’B’’ B’B’’

i j

Base ExchangeTheorem:

B’-i+j and B’’-j+i are both bases

Merging two Bases

B’ B’’

B’B’’ B’B’’

i j

prob p

prob 1-p

B’ B’’

B’B’’ B’B’’

i i

B’ B’’

B’B’’ B’B’’j j

p

p

1-p

1-p

Merging Spanning Trees

0.3

0.6

Merging Spanning Trees

0.3

0.6

0.3

0.6

0.3

0.6

0.3/(0.3+0.6)

0.6/(0.3+0.6)

Swap Rounding for Matroids

Theorem: Randomized-Swap-Rounding with x B(M) outputs a random base B such that• Pr[i B] = xi for each i N• Xi (indicator for i B ) variables are negatively

correlated

Negative correlation gives concentration bounds for linear functions of the Xi s

Swap Rounding for Matroids

Theorem: Randomized-Swap-Rounding with x B(M) outputs a random base B such that• Pr[i B] = xi for each i N• Xi (indicator for i B ) variables are negatively

correlated

Additional properties for submodular functions:• E[f(B)] ≥ F(x) where F is multilinear extension of f• Pr[ f(B) < (1-δ) F(x)] ≤ exp(- F(x) δ2/8)

(concentration for lower tail of submod functions)

Several ApplicationsCan handle matroid constraint plus packing

constraintsx B(M) and Ax ≤ b

• (1-1/e) approximation for submodular functions subject to a matroid plus O(1) knapsack/packing constraints (or many “loose” packing constraints)

• Simpler rounding and proof for “thin” spanning trees in ATSP application ([Asadpour etal’10])

• ...

Proof idea for Negative Correlation

Process is a vector-valued martingale:• each iteration merges two bases• merging bases involves swapping elements

in each step

In each step only two elements i and j involved

Proof idea for Negative Correlation

In each step only two elements i and j involved

Xi, Xj before swap step and X’i, X’j after swap step

1. E[X’i | Xi, Xj ] = Xi and E[X’j | Xi, Xj ] = Xj

2. X’i + X’j = Xi + Xj

Proof idea for Negative Correlation

In each step only two elements i and j involved

Xi, Xj before swap step and X’i, X’j after swap step

1. E[X’i | Xi, Xj ] = Xi and E[X’j | Xi, Xj ] = Xj

2. X’i + X’j = Xi + XjE[X’iX’j|Xi,Xj ] = ¼ E[(X’i+X’j)2| Xi,Xj ] − ¼ E[(X’i - X’j)2| Xi,Xj ]

= ¼ (Xi+Xj)2 − ¼ E[(X’i - X’j)2| Xi, Xj ]

≤ ¼ (Xi+Xj)2 − ¼ (Xi - Xj)2

≤ Xi Xj

Beyond matroids?

Question: Can we obtain negative correlation for other combinatorial structures/polytopes?

Beyond matroids?

Question: Can we obtain negative correlation for other combinatorial structures/polytopes?

Answer: No. Negative correlation implies the polytope is

“essentially” a matroid base polytope

Other Comments

• Swap rounding advantage: • identifies exchange property as the key• Idea generalizes/inspires work for other

structures such as matroid intersection, and b-matchings with some restrictions

• Lower tail for submodular functions uses martingale analysis (does not follow from negative correlation)

• Negative correlation not needed for concentration

Do we need negative correlation for concentration?

No. • Lower tail for submodular functions shown

via martingale method• Also can show concentration for linear

functions in the matroid intersection polytope and non-bipartite matching (a the loss of a bit in expectation)

Example: Rounding in bipartite-matching polytopexe = ½ on each edge Can we round x to a matching?

Example: Rounding in bipartite-matching polytopexe = ½ on each edge Can we round x to a matching?

If we want to preserve expectation of x only choice is to pick one of two perfect matchings, each with prob ½

Large positive correlation!

Informal StatementsFor any point x in the bipartite matching polytope• Can round x to a matching preserving expectation

and negative correlation holds for edge variables incident to any vertex [Srinivasan’99]

• Can round x to a matching x’ s.t E[x’] = (1-γ) x and concentration holds for any linear functions of x (the exponent in tail bound depends on γ) [CVZ]

• Above results generalize to matroid intersection and non-bipartite matchings [CVZ]

Questions?

Thanks!

Submodular Functions

• Non-negative submodular set functionsf(A) ≥ 0 for all A

• Monotone submodular set functionsf(ϕ) = 0 and f(A) ≤ f(B) for all A B

• Symmetric submodular set functionsf(A) = f(N\A) for all A

Multilinear Extension of f

[CCPV’07] inspired by [Ageev-Sviridenko]For f: 2N R+ define F:[0,1]N R+ as

x = (x1, x2, ..., xn) [0,1]|N|

F(x) = Expect[ f(x) ] = S N f(S) px(S)

= S N f(S) i S xi i N\S (1-xi)

Multilinear Extension of f

For f: 2N R+ define F:[0,1]N R+ as

F(x) = S N f(S) i S xi i N\S (1-xi)

F is smooth submodular ([Vondrak’08])• F/xi ≥ 0 for all i (monotonicity)

• 2F/xixj ≤ 0 for all i,j (submodularity)

Optimizing F(x)

[Vondrak’08]Theorem: For any down-monotone polytope

P [0,1]n max F(x) s.t x P can be optimized to within a (1-1/e) approximation if we can do linear optimization over P

Algorithm: Continuous-Greedy