Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. ·...
Transcript of Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. ·...
![Page 1: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/1.jpg)
Submodular Optimization and Approximation Algorithms
Satoru Iwata (RIMS, Kyoto University)
![Page 2: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/2.jpg)
Submodular Functions
• Cut Capacity Functions • Matroid Rank Functions • Entropy Functions
Finite Set
![Page 3: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/3.jpg)
Entropy Functions
Information Sources
Entropy of the Joint Distribution
Conditional Mutual Information
![Page 4: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/4.jpg)
Positive Definite Symmetric Matrices
Ky Fan’s Inequality
Extension of the Hadamard Inequality
![Page 5: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/5.jpg)
Discrete Concavity
Diminishing Returns
![Page 6: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/6.jpg)
Discrete Convexity
Convex Function
![Page 7: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/7.jpg)
Discrete Convexity Lovász (1983)
: Linear Interpolation
: Convex
: Submodular
Murota (2003)
Discrete Convex Analysis
![Page 8: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/8.jpg)
Submodular Function Minimization
Minimization Algorithm
Evaluation Oracle
Minimizer
Assumption:
Ellipsoid Method Grötschel, Lovász, Schrijver (1981)
![Page 9: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/9.jpg)
Submodular Function Minimization Grötschel, Lovász, Schrijver (1981, 1988)
Iwata, Fleischer, Fujishige (2000) Schrijver (2000)
Iwata (2003)
Fleischer, Iwata (2000)
Orlin (2007)
Iwata (2002)
Fully Combinatorial
Ellipsoid Method
Cunningham (1985)
Iwata, Orlin (2009)
![Page 10: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/10.jpg)
Symmetric Submodular Functions
Symmetric
Symmetric Submodular Function Minimization
Minimum Cut Algorithm by MA-ordering Nagamochi & Ibaraki (1992) Minimum Degree Ordering Nagamochi (2007)
Queyranne (1998)
![Page 11: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/11.jpg)
Application to Clustering
Minimize
subject to
Greedy Split
-Approximation
Monotone, Submodular
Zhao, Nagamochi, Ibaraki (2005)
![Page 12: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/12.jpg)
Submodurlar Function Maximization
Approximation Algorithms Nemhauser, Wolsey, Fisher (1978) Monotone SF Cardinality Constraint (1-1/e)-Approximation
Vondrák (STOC 2008) Monotone SF Matroid Constraint (1-1/e)-Approximation
![Page 13: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/13.jpg)
Submodurlar Function Maximization
Approximation Algorithms Feige, Mirrokni, Vondrák (FOCS 2007) Nonnegative SF 2/5-Approximation
Lee, Mirrokni, Nagarajan, Sviridenko (STOC 2009) Nonnegative SF 1/4-Approximation (Matroid Constraint) 1/5-Approximation (Knapsack Constraints)
![Page 14: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/14.jpg)
Feature Selection Random Variables
Predict from subset “Sick”
“Fever” “Rash” “Cough” Maximize subject to Conditionally
Independent
Submodular
Krause & Guestrin (2005)
![Page 15: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/15.jpg)
Submodular Welfare Problem
Monotone, Submodular
Utility Functions
Maximize
subject to
-Approximation Vondrák (2008)
![Page 16: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/16.jpg)
Algorithm with for monotone submodular functions
Construct a set function such that
For what function is this possible?
Approximating Submodular Functions Goemans, Harvey, Iwata & Mirrokni (2009)
![Page 17: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/17.jpg)
Question What Kind of Approximation Algorithms Can Be Extended to Optimization Problems
with Submodular Cost or Constraints ?
Cf. Submodular Flow (Edmonds & Giles, 1977)
Svitkina & Fleischer (FOCS 2008) Sampling Algorithms, Lower Bounds Submodular Sparsest Cut Submodular Load Balancing
![Page 18: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/18.jpg)
Question How Can We Exploit Discrete Convexity
in Design of Approximation Algorithms?
Cf. Ellipsoid Method (Grötschel, Lovász & Schrijver, 1981)
Chudak & Nagano (SODA 2007) Facility Location with Submodular Penalty
![Page 19: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/19.jpg)
Submodular Function Minimization under Covering Constraints
• Submodular Vertex Cover Problem Rounding Algorithm • Submodular Cost Set Cover Problem Rounding Algorithm Primal-Dual Algorithm • Submodular Edge Cover Problem Inapproximability
Iwata & Nagano (2009)
![Page 20: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/20.jpg)
The Vertex Cover Problem Graph
Vertex Cover
Find a Vertex Cover
Minimizing
![Page 21: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/21.jpg)
The Vertex Cover Problem Integer Programming Formulation
Minimize
subject to
Linear Programming Relaxation (LPR)
![Page 22: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/22.jpg)
The Vertex Cover Problem
Lemma (Nemhauser & Trotter, 1974)
LPR has a half-integral optimal solution.
Every nonsingular submatrix in the coefficient matrix has a half-integral inverse.
![Page 23: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/23.jpg)
The Vertex Cover Problem
Optimal Solution of LPR
Vertex Cover
Vertex Cover
2-Approximation Solution
![Page 24: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/24.jpg)
Submodular Vertex Cover Graph
Submodular
Find a Vertex Cover Minimizing
The Vertex Cover Problem
![Page 25: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/25.jpg)
Relaxation Problem
Convex Programming Relaxation (CPR)
Minimize
subject to
Lemma
CPR has a half-integral optimal solution.
![Page 26: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/26.jpg)
Proof of Half-Integrality
Every nonsingular submatrix in the coefficient matrix has a half-integral inverse.
![Page 27: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/27.jpg)
Rounding Algorithm Half-Integral Optimal Solution of CPR
Vertex Cover
2-Approximation Solution
Vertex Cover
![Page 28: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/28.jpg)
How to Solve CRP
Find a Vertex Cover
Minimizing
in
![Page 29: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/29.jpg)
How to Solve CPR
Vertex Cover
Optimal Vertex Cover with
![Page 30: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/30.jpg)
How to Solve CPR Vertex Cover with
Feasible to CPR
![Page 31: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/31.jpg)
Lower Bounds Goel, Karande, Tripathi, Wang (FOCS 2009)
No polynomial algorithm can solve the submodular vertex cover problem approximately within a factor smaller than 2.
Rounding Algorithm Using Ellipsoid Method for Monotone Submodular Functions
![Page 32: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/32.jpg)
Submodular Cost Set Cover Find Covering with Minimum
Rounding Algorithm -Approximation
Primal-Dual Algorithm
Greedy -Approximation Algorigm for Monotone Submodular Functions
Koufogiannakis & Young (ICALP 2009)
![Page 33: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/33.jpg)
Relaxation Problem
Minimize
(SCP)
subject to
Ellipsoid Method
![Page 34: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/34.jpg)
Rounding Algorithm
Optimal Solution of (SCP)
Set Cover
-Approximation Solution
Minimizer of among Supersets of
![Page 35: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/35.jpg)
Rounding Algorithm
Set Cover
Set Cover
Monotone, Submodular
![Page 36: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/36.jpg)
Primal-Dual Approximation
Maximize
subject to
Dual Problem of (SCP)
Submodular Polyhedron
![Page 37: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/37.jpg)
Primal-Dual Approximation Initially,
Unique Maximal Set with
![Page 38: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/38.jpg)
Primal-Dual Approximation
Set Cover
Set Cover
-Approximation
![Page 39: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/39.jpg)
Submodular Cost Set Cover What if is not bounded?
For each Minimizer of Covering Set Cover
Set Cover
-Approximation
Greedy -Approx.?
![Page 40: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/40.jpg)
Submodular Edge Cover Graph
Edge Cover
Submodular, Nonnegative,
Find an Edge Cover Minimizing
NP-hard MIN 2SAT
![Page 41: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/41.jpg)
Inapproximability Result
Theorem No oracle-polynomial algorithm can solve the submodular edge cover problem approximately within a factor of
![Page 42: Submodular Optimization and Approximation Algorithmstetali/LINKS/IWATA/SOAA.pdf · 2009. 8. 18. · Satoru Iwata (RIMS, Kyoto University) Submodular Functions • Cut Capacity Functions](https://reader035.fdocuments.in/reader035/viewer/2022071517/613ac998f8f21c0c8268a298/html5/thumbnails/42.jpg)
Summary Submodular Vertex Cover
CP-Rounding
2-Approx.
No Oracle-Polynomial Algorithm with an Approximation Ratio.
Submodular Cost Set Cover
CP-Rounding
Primal-Dual -Approx.
Submodular Edge Cover
-Approx.