Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)
-
Upload
daniel-bradford -
Category
Documents
-
view
216 -
download
0
Transcript of Satyen Kale (Yahoo! Research) Joint work with Sanjeev Arora (Princeton)
A Combinatorial, Primal-Dual Approach to Semidefinite Programming
Satyen Kale (Yahoo! Research)
Joint work withSanjeev Arora (Princeton)
Semidefinite Programming
Semidefinite Program (SDP): find X s.t.A1 ² X · 0A2 ² X · 0
Am ² X · 0
X º 0 Tr(X) = 1
All eigenvalues are non-negative´ 9 v1, v2, …, vn s.t. Xij = vi ¢ vj
A ² B = Tr(AB) = ij AijBij
Density matrix
Semidefinite ProgrammingMax Cut [GW’95]
Graph Coloring [KMS’98, ACC’06]
(a Ç :b) Æ (:a Ç c) Æ (a Ç b) Æ (:c Ç b)
Constraint Satisfaction [ACMM’05, R ’08, RS ‘09,..]1 0 0 0 1 1 0 1
SDP
Balanced Partitioning [ARV’04]
Control Theory
Algorithms for SDP
Ellipsoid method [GLS’81]:• O(n8) iterations
Interior point methods [NN’90, A’95]:• O(√n ¢ m3) time
Lagrangian Relaxation [KL’95], [AHK’05]:• Reduction to eigenvectors• poly(1/) dependence on , limits applicability
A1 ² X · 0 An ² X · 0
i wiAi ² X · 0
Combinatorial, Primal-Dual algorithms
(analogous to flow algorithms via LP)
Challenges in designing efficient, combinatorial algorithms for SDP
Positive semidefiniteness hard to maintain
Rounding algorithms exploit geometric structure (e.g. negative-type metrics)
Matrix operations inefficient to implement
Our Work
A general scheme that yields fast,
combinatorial, primal-dual approximation algorithms for various
combinatorial optimization problems using SDP relaxations
Our Results: Primal-Dual algorithms
Problem Previous best:
O(√log n) apx
Our algorithm:O(√log n)
apx
Our algorithm:
O(log n) apx
UndirectedSparsest Cut
Õ(n2)[AHK’04]
Õ(n2) Õ(m + n1.33)
UndirectedBalanced Sep.
Õ(n2)[AHK’04]
Õ(n2) Õ(m + n1.33)
DirectedSparsest Cut
Õ(n9.5) Õ(m1.33 + n2) Õ(m1.33)
DirectedBalanced Sep.
Õ(n9.5) Õ(m1.33 + n2) Õ(m1.33)
Min UnCut Õ(n9.5) Õ(n3) ---
Min 2CNF Deletion
Õ(n9.5) Õ(nm1.33 + n3)
---
Õ(m + n1.33) [S’09]
Õ(m + n1.33) [S’09]Unknown how to achieve using LP!
Near-linear time alg for MAXCUT
Also yields Õ(m) time algorithm for approximating MAXCUT SDP Gives first near-linear implementation of
Goemans-Williamson approximation algorithm
Previous best: Õ(nm) [KL’95]
LP via Multiplicative Weights [L’88]
a1 ¢ x · 0a2 ¢ x · 0
am ¢ x · 0
x ¸ 0i xi = 1
Multiplicative WeightsInitialize: x1 = (1/n, …, 1/n)Update:xt+1 = xt £ exp(- ait
)/t
t = normalization factor
Violation CheckerFind it s.t. ait
¢ xt >
xt
ait
Thm: Finds -feasible x in O(2log(n)/2) iterations.
= maxij |aij| (“width”)
Convex combination of constraints allowed:i yi ai ¢ xt > , where yi ¸ 0, i yi = 1.
Hence Primal-Dual.
Analysis uses t as potential function
Coordinate-wise
a1 ¢ x · a2 ¢ x ·
am ¢ x ·
x ¸ 0i xi = 1
SDP via Matrix Multiplicative Weights
A1 ² X · 0A2 ² X · 0
Am ² X · 0
X º 0Tr(X) = 1
Matrix MWInitialize: X1 = (1/n)IUpdate:Xt+1 = exp(- t
s=1 Ais)/t
t = normalization factor
Violation CheckerFind it s.t. Ait
² Xt >
Xt
Ait
Thm: Finds -feasible X in O(2log(n)/2) iterations.
= maxi kAik (“width”)
Convex combination of constraints allowed:i yi Ai²Xt > , where yi ¸ 0, i yi = 1.
Hence Primal-Dual.
Analysis uses t as potential function
The Matrix Exponential Matrix exponential:exp(A) = I + A + A2/2! +
A3/3! + …
Always PSD: exp(A) º 0
Tricky beast:exp(A+B) = exp(A) exp(B)
Computation: O(n3) time Computing exp(A)v:
Õ(m) time (arises as soln of ODE)
Golden-Thompson inequality:Tr(exp(A+B)) · Tr(exp(A)exp(B)) Matrix MW
Initialize: X1 = (1/n)IUpdate:Xt+1 = exp(- t
s=1 Ais)/t
t = normalization factor
Approximation via SDP Relaxations
Input
QuadraticProgram
SDP:Vector
variables
Reduction
SDP Solver
RoundingAlgorithm
SDP Opt: XXij = vi ¢ vj
v1
vn
v2
RelaxationPrimal-Dual SDP
Primal-Dual SDP Framework
Reduction
RelaxationRoundingAlgorithm
v1
vn
v2
MatrixMWX
y1,…, ym
Primal-Dual SDP
SDP:Vector
variables
Input
Xij = vi ¢ vj
QuadraticProgram
Approximating Balanced Separator
Cut (S, S’) is c-balanced if |S|, |S’| ¸ cn
Min c-Balanced Separator: c-balanced cut of min
capacity
Numerous applications: Divide-and-conquer algs Markov chains Geometric embeddings Clustering Layout problems …
G = (V, E)
S
S’
SDP for Balanced Separator
¼kvi – vjk2 = 0 if i, j on same side,
= 1 otherwise
SDPmin i,j 2 E ¼kvi – vjk2
8i: kvik2 = 1
i, j ¼kvi – vjk2 ¸ c(1-c)n2
8ijk: kvi–vjk2 + kvj–vkk2 ¸ kvi–vkk2
vi = -1 vi = 1
G = (V, E)
S
S’
Finding Violated Constraints
SDP(C1) i,j 2 E ¼kvi – vjk2 <
(C2) 8i: kvik2 = 1
(C3) i, j ¼kvi – vjk2 ¸ c(1-c)n2
(C4) 8ijk: kvi–vjk2 + kvj–vkk2 ¸ kvi–vkk2
To find violated constraints, given X (thus, vi): Check (C2) and (C3) To check (C1) and (C4): find multicommodity
flow s.t.1. Total flow from any node is at most d = Õ(/n)2. Total flow on any edge is at most 13. i,j fijkvi – vjk2 ¸ 4
fij = total flow between i, j
Bounds width
Binary search parameter
Either (C1) or (C4) violated
Finding flow: O(log n) approx
Xij = vi ¢ vj
v1
vn
v2
d
d
d
d
d
d
d
To find flow s.t. ij fijkvi – vjk2 ¸ 4
Thm: This yields either:1. a flow s.t. ijfijkvi – vjk2 ¸ 42. a cut of value O(log(n))
kvik2 = 1ij ¼kvi – vjk2 ¸ c(1-c)n2
Max flow
Xij = vi ¢ vj
v1
vn
v2
Thm: This yields either:1. a flow s.t. ijfijkvi – vjk2 ¸ 42. a cut of value O(√log(n))
d
d
dd
dd
d
d
Multicommodity flow
To find flow s.t. ij fijkvi – vjk2 ¸ 4kvik2 = 1ij ¼kvi – vjk2 ¸ c(1-c)n2
Finding flow: O(plog n) approx
Other Applications of Matrix MWDerandomization• Deterministic Expander Graphs Constructions [K’07, WX
’07]• Deterministic Approximation via Covering SDPs [K ’07, WX
’07]Quantum Computing• QIP = PSPACE [JJUW ’10] and related results [JUW ’09, JW
’09]• Learnability of Quantum States [A ’06, K ’07]
Machine Learning• Online Variance Minimization [WK ’06]• Online PCA [WK ’06]
Other SDP based Approximation Algorithmss• Near Linear-Time Approximation Algorithms for 2-CSPS [S
’10]• Near-Linear Time Balanced Graph Partitioning [OV ’11]
Future Work
Make SDP solver efficient in practice Sampling, rescaling techniques might be
useful ``Combinatorialize’’ interior-point
methods?
Near linear-time algs for approx max flow?
Thank You!