Sequential and Parallel Algorithms for Mixed Packing and...
44
Sequential and Parallel Algorithms for Mixed Packing and Covering (from FOCS 2001) Neal E. Young UC Riverside
Transcript of Sequential and Parallel Algorithms for Mixed Packing and...
Sequential and Parallel Algorithms for Mixed Packing and Covering
(from FOCS 2001)
Neal E. Young
UC Riverside
2
Marek is being punished.
He can eat only bacon, beans, and beets!
Can Marek get enough of what he needswithout getting too much of what he doesn’t?
3
Nutrition FactsServing Size 4 pc
Servings per container 10
Amount Per Serving
Calories 110 Calories from Fat 80
% Daily Value
Total Fat 10g 30 % Saturated Fat 9g 50 %
Cholesterol 3mg 10 %
Sodium 500mg 40 %
Total Carbohydrates 0g 0 %
Dietary Fiber 0g 0 %
Sugars 0g
Protein 5g 30 %
Vitamin A 0% Vitamin B 7%
Vitamin C 2% Iron 16%
BaconNutrition FactsServing Size 1 cup
Servings per container 2
Amount Per Serving
Calories 200 Calories from Fat 16
% Daily Value
Total Fat 2g 5 % Saturated Fat 0g 0 %
Cholesterol 0mg 0 %
Sodium 0 mg 0 %
Total Carbohydrates 6g 20 %
Dietary Fiber 2g 15 %
Sugars 0g
Protein 6g 35 %
Vitamin A 40% Vitamin B 0%
Vitamin C 22% Iron 2%
BeansNutrition FactsServing Size 3 oz
Servings per container 5
Amount Per Serving
Calories 180 Calories from Fat 0
% Daily Value
Total Fat 0g 0 % Saturated Fat 0g 0 %
Cholesterol 3mg 15 %
Sodium 30mg 2 %
Total Carbohydrates 8g 30 %
Dietary Fiber 2g 15 %
Sugars 6g
Protein 0g 0 %
Vitamin A 40% Vitamin B 52%
Vitamin C 26% Iron 3%
Beets
4
protein: 30 bacon + 35 bean > 100
vitamin A: 40 bean + 43 beet > 100
vitamin B: 7 bacon + 52 beet > 100
vitamin C: 2 bacon + 22 bean + 26 beet > 100
fat: 30 bacon + 5 bean < 100
sugar: 15 bean + 37 beet < 100
salt: 40 bacon + 2 beet < 100
cholesterol: 10 bacon + 10 bean + 15 beet < 100
constraints
unknowns
bacon, bean, beet1 serving beans has 35%of the RDA of protein
5
protein: 30 bacon + 35 bean > (1-ε)100 vitamin A: 40 bean + 43 beet > (1-ε)100 vitamin B: 7 bacon + 52 beet > (1-ε)100 vitamin C: 2 bacon + 22 bean + 26 beet > (1-ε)100
fat: 30 bacon + 5 bean < (1+ε)100sugar: 15 bean + 37 beet < (1+ε)100 salt: 40 bacon + 2 beet < (1+ε)100
cholesterol: 10 bacon + 10 bean + 15 beet < (1+ε)100
ε-approximate solutions:
6
Bibliography
[1950] von Neumann. Numerical method for determination of the value and the best strategies of a zero-sum two-person game with large numbers of strategies.
[1950] Brown and von Neumann. Solutions of games by differential equations.
[1952] Chernoff. A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations. --- Chernoff bound, implicit use of Lmax-like functions
[1958] Ford and Fulkerson. A suggested computation for maximal multicommodity flow.
[1960] Dantzig and Wolfe. Decomposition principle for linear programs.
[1962] Benders. Partitioning procedures for solving mixed-variables programming problems.
[1971] Held and Karp. The traveling salesman problem and minimum spanning trees.
[1977] Khachiyan. Convergence rate of the game processes for solving matrix games. ...[1979] Shapiro. A survey of Lagrangean techniques for discrete optimization. Annals of Discrete Mathematics, 5:113--138, 1979.
7
BibliographyGrigoriadis and Khachiyan. A sublinear-time randomized approximation algorithm for matrix games. OR Research Letters, 1995. An exponential-function reduction method for block-angular convex programs. Networks, 1995.
Young. Randomized rounding without solving the linear program. SODA, 1995.
Karger and Plotkin. Adding multiple cost constraints to combinatorial optimization problems, with applications to multicommodity flows. STOC, 1995.
Grigoriadis and Khachiyan. Coordination complexity of parallel price-directive decomposition. MOR, 1996. Approximate minimum-cost multicommodity flows in o(knm/ε^2) time. Math. Programming, 1996.
Garg and Konemann. Faster and simpler algorithms for multicommodity flow and other fractional packing problems. FOCS, 1998. --- variable-size increments
Konemann. Fast Combinatorial Algorithms for Packing and Covering ProblemsPhD thesis, Max-Planck-Institute for Informatik, 2000. --- dropping met covering constraints
Fleischer. Approximating fractional multicommodity flow independent of the number of commodities. SIAM J. Discrete Math, 2000. --- partitioning increments into phases
8
30 bacon + 35 bean > 100
40 bean + 43 beet > 100
7 bacon + 52 beet > 100
2 bacon + 22 bean + 26 beet > 100
30 bacon + 5 bean < 100
15 bean + 37 beet < 100
40 bacon + 2 beet < 100
10 bacon + 10 bean + 15 beet < 100
linear program
fat
sugar
salt
chol.
vit A
vit B
vit C
prot.
beets
?
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
beans
?
bacon
?
9
fat?
sugar?
salt?
chol?
vit A?
vit B?
vit C?
prot?
beets
1
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
beans
0
bacon
0
One serving of beets?
10
0%
37%
2%
15%
43%
52%
26%
0%
beets1
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
beans0
bacon0
After one serving of beets
RDA vit A
RDA vit B
RDA vit C
RDA prot.RDA fat
RDA sugar
RDA salt
RDA chol.
11
0%
37%
2%
15%
43%
52%
26%
0%
beets1
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
beans0
bacon0
get protein
avoid sugar
Greedy approach: Get protein, avoid sugar … … eat bacon
min(…)0
max(…)
37
12
30
37
42
25
43
59
28
30
beets1
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
beans0
bacon1
get vitamin C
avoid salt
Get vitamin C, avoid salt … … eat beets
min(…)28
max(…)
42
13
30
74
44
40
86
111
54
30
beets2
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
beans0
bacon1
Protein low
Sugar high
Get protein, avoid sugar … … eat bacon
min(…)30
max(…)
74
14
60
74
84
50
86
118
56
60
beets2
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
beans0
bacon2
Vitamin C low
Salt high
Get vitamin C, avoid salt … … eat beans
min(…)56
max(…)
84
15
65
89
84
60
126
118
78
95
beets2
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
beans1
bacon2
vitamin C low
Sugar high
Get vitamin C, avoid sugar … … eat beans?
min(…)78
max(…)
89
16
69
100
84
67
156
118
94.5
121
beets2
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
beans1.75
bacon2
vitamin C low
Get vitamin C, completely avoid sugar… … eat bacon?
sugar maxed out
min(…)94.5
max(…)
100
17
80.7
100
100
71
156
121
95.3
133
beets2
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
beans1.75
bacon2.4
vitamin C low
Get vitamin C, completely avoid sugar and salt … … stuck!
sugar maxed out
salt maxed out
min(…)96.6
max(…)
100
18
Making a greedy approach work
Balance all needs in each step.
Take small bites.
19
Balancing all needs…
Smooth approximations of Max() and Min()
20
Lmax(x,-x)
Max(x,-x)
21
lmax < max + ln m
ln(m)
22
Change in Lmax() when inputs change:
gradient estimate is ε-approximate within ε-neighborhood
+ε
Lmax
trueg
true < (1+ε)g
g = change estimated by gradient
23
0
0
0
0
0
0
0
0
beets0
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
beans0
bacon0
Algorithm: use Lmin and Lmax instead of min and max.Choose increments so Lmin increases by > (1-ε) times as much as Lmax.
Lmin(…)-1.38
Lmax(…)1.38
initially ln m
24
Choose increments so Lmin increases by > (1-ε) times as much as Lmax.
Stop when Lmin > ln(m)/ε (= 13.8)
to get ε-approximate solution:
30 bacon + 35 bean > ln(m)/ε 40 bean + 43 beet > 13.8
7 bacon + 52 beet > 13.8
2 bacon + 22 bean + 26 beet > 13.8
30 bacon + 5 bean < ln(m) + [ln(m)/ε + ln(m)]/(1-ε) = (1+O(ε))ln(m)/ε 15 bean + 37 beet < (1+O(ε))13.840 bacon + 2 beet < (1+O(ε))13.810 bacon + 10 bean + 15 beet < (1+O(ε))13.8
target
25
Use gradients to estimate increase in Lmin and Lmax.
0
0
0
0
0
0
0
0
beets+δ ?
30
72
35
40
22
26
5243
30
4010
5
15
10
15
2
37
beans0
bacon0
Lmin(…)+30.2δ
Lmax(…)
+13.5δ
Variable ok to raise if est. increase in lmin > .9 est. increase in lmax
Partial derivative of lmin w.r.t. beets is 30.2
Estimate is ε-accurate if inputs to lmin change < ε
26
0
0
0
0
0
0
0
0
beetsok
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
beans+δ?
bacon0
Beets are ok, what about beans?
Lmin(…)
+24.2δLmax(…)
+7.5δ
27
0
0
0
0
0
0
0
0
beetsok
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
okbeans
bacon+δ?
What about bacon?
Lmin(…)
+9.7δLmax(…)
+20δ
28
0
0
0
0
0
0
0
0
beetsok
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
okbeans
bacon
No!
Beets and beans are okay to raise, but not bacon.
Lmin(…)
-1.38Lmax(…)
1.38
29
Raise beets. Now: by how much? For gradient estimates to be ε-accurate, need inputs to lmin and lmax to change by < ε
0
0
0
0
0
0
0
0
beets+δ ?
30
72
35
40
22
26
5243
30
4010
5
15
10
15
2
37
beans0
bacon0
Lmin(…)+30.2δ
Lmax(…)
+13.5δ
Partial derivative of lmin w.r.t. beets is 30.2
Estimate is ε-accurate if inputs to lmin change < ε
30
0
0.07
0.00
0.03
0.08
0.1
0.05
0
beets.002
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
0beans
bacon
0
Raise beets just enough so some constraint increases by ε. (ε = 0.1)
Lmin(…)
-1.33Lmax(…)
1.41
Up .05 Up .03
Vitamin B up 0.1
31
0
0.07
0.00
0.03
0.08
0.1
0.05
0
beets.002
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
0beans
bacon
0
Repeat. Check variables… and raise ok one so some constraint increases by ε
Lmin(…)
-1.33Lmax(…)
1.41
ok
ok
no
32
0
1.06
0.05
0.43
1.24
1.5
0.75
0
beets.029
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
0beans
bacon
0
Lmin(…)
-0.69Lmax(…)
1.87
no
ok
no
Fifteen (1/520)-servings of beets later…
Up .70 Up .49
33
0.42
2.34
0.06
1.28
4.64
1.5
2.62
3.0
beets.029
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
.085beans
bacon
0
Lmin(…)
1.03Lmax(…)
2.81
no
ok
ok
and thirty-four (1/400)-servings of beans…
Up 2.41 Up 1.42
34
4.65
8.46
5.31
5.39
13.9
8.97
8.25
10.0
beets.155
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
.180beans
bacon.125
Lmin(…)
7.73Lmax(…)
8.57
After 204 rounds: (recall target = 13.8)
35
4.65
8.46
5.31
5.39
13.9
8.97
8.25
10.0
beets.155
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
.180beans
bacon.125
Lmin(…)
7.76Lmax(…)
8.57
Delete covering constraints as they are satisfied
36
335 total rounds until lmin > target:
15 beet 34 bean 5 bacon 17 beet 12 bean 15 bacon 17 beet 12 bean 15 bacon 16 beet 12 bean 15 bacon 16 beet 12 bean 14 bacon 15 beet 13 bean 15 bacon 18 beet 9 bean 18 bacon 8 beet 10 bean
37
64
98
73
66
168
100
100
136
beets1.69
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
2.39beans
bacon1.74
At end, scale up by 100/target:
38
Algorithm1. x = (0,0,…,0)2. target = ln(m)/ε3. Until lmin > target do:4. Let x = x + δ, where vector δ satisfies:5. δ[j]>0 only if ok to raise variable x[j],6. and some constraint increases by ε.7. Delete any satisfied covering constraints.8. Return x, appropriately scaled.
39
Correctness Is there always a variable to raise?
Does the algorithm terminate?
40
Lemma: If the constraints can be satisfied,
there is always a variable to raise. Suppose there is a feasible solution x*:
<100
<100
<100
<100
>100
>100
>100
>100
x[1]
x*[1]
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
x*[2]
x[2]
x[3]
x*[3]
At most 100
At least 100
41
Lemma: If the constraints can be satisfied,
there is always a variable to raise. Take vector δ = ε x*/100.
<ε
<ε
<ε
<ε
>ε
>ε
>ε
>ε
x[1]
δ[1]
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
δ[2]
x[2]
x[3]
δ[3]
42
Lemma: If the constraints can be satisfied,
there is always a variable to raise. Add δ to current x. Lmin increases by as much as lmax.
+ε
+ε
+ε
+ε
+ε
+ε
+ε
+ε
x[1]
x+δ[1]
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
x+δ[2]
x[2]
x[3]
x+δ[3]
Lmin(…)+ε
Lmax(…)+ε
Raise by εor less
Raise by εor more
QED
43
Lemma: There are O(m log(m)/ε ) increments
Each increment increases at least one constraint by ε.
+?
+?
+?
+ε
+?
+?
+?
+?
x[1]
+δ[1]
30
72
35
40
22
26
52
43
30
40
10
5
15
10
15
2
37
+δ[2]
x[2]
x[3]
+δ[3]
Constraint < O(target) at end,
so at most O(target/ε) increments per constraint.
target = O(log(m)/ε)
2
QED
44
Summary
Solutions meet constraints within 1+O(ε) factor.
O(m log(m)/ε ) linear-time increments.
Faster implementations possible.
2
