The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤...
Transcript of The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤...
![Page 1: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/1.jpg)
The Knapsack Problem
W20
1020
15
n items with weight wi ∈ N and profit pi ∈ N
Choose a subset x of items
Capacity constraint∑
i∈x wi ≤ Wwlog assume
∑i wi > W , ∀i : wi < W
Maximize profit∑
i∈x pi
– 28. April 2010 1/44
![Page 2: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/2.jpg)
Optimization problem
Set of instances I
Function F that gives for all w ∈ I the set of feasiblesolutions F (w)
Goal function g that gives for each s ∈ F (w) the value g(s)
Optimization goal: Given input w , maximize or minimize thevalue g(s) among all s ∈ F (w)Decision problem: Given w ∈ I and k ∈ N, decide whether
OPT (w) ≤ k (minimization)
OPT (w) ≥ k (maximization)
where OPT (w) is the optimal function value among alls ∈ F (w).
– 28. April 2010 2/44
![Page 3: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/3.jpg)
Optimization problem
Set of instances I
Function F that gives for all w ∈ I the set of feasiblesolutions F (w)
Goal function g that gives for each s ∈ F (w) the value g(s)
Optimization goal: Given input w , maximize or minimize thevalue g(s) among all s ∈ F (w)Decision problem: Given w ∈ I and k ∈ N, decide whether
OPT (w) ≤ k (minimization)
OPT (w) ≥ k (maximization)
where OPT (w) is the optimal function value among alls ∈ F (w).
– 28. April 2010 2/44
![Page 4: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/4.jpg)
Quality of approximation algorithms
Recall: An approximation algorithm A producing a solution ofvalue A(w) on a given input w ∈ I has approximation ratio r iff
A(w)
OPT (w)≤ r ∀w ∈ I
(for maximization problems) or
OPT (w)
A(w)≤ r ∀w ∈ I
(for minimization problems)How good your approximation algorithm is depends on thevalue of r and its running time.
– 28. April 2010 3/44
![Page 5: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/5.jpg)
Negative result
We cannot find a result with bounded difference to the optimalsolution in polynomial time.Interestingly, the problem remains NP-hard if all items have thesame weight to size ratio!
– 28. April 2010 4/44
![Page 6: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/6.jpg)
Reminder?: Linear Programming
DefinitionA linear program with n variables and m constraints is specifiedby the following minimization problem
Cost function f (x) = c · xc is called the cost vector
m constraints of the form ai · x ./i bi where ./i∈ {≤,≥,=},ai ∈ R
n We have
L ={
x ∈ Rn : ∀1 ≤ i ≤ m : xi ≥ 0 ∧ ai · x ./i bi
}.
Let aij denote the j-th component of vector ai .
– 28. April 2010 5/44
![Page 7: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/7.jpg)
Reminder?: Linear Programming
DefinitionA linear program with n variables and m constraints is specifiedby the following minimization problem
Cost function f (x) = c · xc is called the cost vector
m constraints of the form ai · x ./i bi where ./i∈ {≤,≥,=},ai ∈ R
n We have
L ={
x ∈ Rn : ∀1 ≤ i ≤ m : xi ≥ 0 ∧ ai · x ./i bi
}.
Let aij denote the j-th component of vector ai .
– 28. April 2010 6/44
![Page 8: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/8.jpg)
Complexity
TheoremA linear program can be solved in polynomial time.
Worst case bounds are rather high
The algorithm used in practice (simplex algorithm) mighttake exponential worst case time
Reuse is not only possible but almost necessary
– 28. April 2010 7/44
![Page 9: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/9.jpg)
Integer Linear Programming
ILP: Integer Linear Program, A linear program with theadditional constraint that all the xi ∈ Z
Linear Relaxation: Remove the integrality constraints from anILP
– 28. April 2010 8/44
![Page 10: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/10.jpg)
Example: The Knapsack Problem
maximize p · x
subject to
w · x ≤ W , xi ∈ {0,1} for 1 ≤ i ≤ n.
xi = 1 iff item i is put into the knapsack.0/1 variables are typical for ILPs
– 28. April 2010 9/44
![Page 11: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/11.jpg)
Linear relaxation for the knapsackproblem
maximize p · x
subject to
w · x ≤ W , 0 ≤ xi ≤ 1 for 1 ≤ i ≤ n.
We allow items to be picked “fractionally”x1 = 1/3 means that 1/3 of item 1 is put into the knapsackThis makes the problem much easier. How would you solve it?
– 28. April 2010 10/44
![Page 12: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/12.jpg)
The Knapsack Problem
W20
1020
15
n items with weight wi ∈ N and profit pi ∈ N
Choose a subset x of items
Capacity constraint∑
i∈x wi ≤ Wwlog assume
∑i wi > W , ∀i : wi < W
Maximize profit∑
i∈x pi
– 28. April 2010 11/44
![Page 13: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/13.jpg)
How to Cope with ILPs
− Solving ILPs is NP-hard
+ Powerful modeling language
+ There are generic methods that sometimes work well
+ Many ways to get approximate solutions.
+ The solution of the integer relaxation helps. For examplesometimes we can simply round.
– 28. April 2010 12/44
![Page 14: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/14.jpg)
Linear Time Algorithm forLinear Relaxation of Knapsack
Classify elements by profit densitypi
wiinto B, {k},S such that
∀i ∈ B, j ∈ S :pi
wi≥
pk
wk≥
pj
wj, and,
∑
i∈B
wi ≤ W but wk +∑
i∈B
wi > W .
Set xi =
1 if i ∈ BW−
∑i∈B wi
wkif i = k
0 if i ∈ S
1015
20
20
WB
S
k
– 28. April 2010 13/44
![Page 15: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/15.jpg)
xi =
1 if i ∈ BW−
∑i∈B wi
wkif i = k
0 if i ∈ S
Lemmax is the optimal solution of the linear relaxation.
Proof.Let x∗ denote the optimal solution
w · x∗ = W otherwise increase some xi
∀i ∈ B : x∗
i = 1 otherwiseincrease x∗
i and decrease some x∗
j for j ∈ {k} ∪ S∀j ∈ S : x∗
j = 0 otherwisedecrease x∗
j and increase x∗
k
This only leaves xk =W−
∑i∈B wi
wk
1015
20
20
WB
S
k
– 28. April 2010 14/44
![Page 16: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/16.jpg)
xi =
1 if i ∈ BW−
∑i∈B wi
wkif i = k
0 if i ∈ S
Lemmax is the optimal solution of the linear relaxation.
Proof.Let x∗ denote the optimal solution
w · x∗ = W otherwise increase some xi
∀i ∈ B : x∗
i = 1 otherwiseincrease x∗
i and decrease some x∗
j for j ∈ {k} ∪ S∀j ∈ S : x∗
j = 0 otherwisedecrease x∗
j and increase x∗
k
This only leaves xk =W−
∑i∈B wi
wk
1015
20
20
WB
S
k
– 28. April 2010 14/44
![Page 17: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/17.jpg)
xi =
1 if i ∈ BW−
∑i∈B wi
wkif i = k
0 if i ∈ S
Lemmax is the optimal solution of the linear relaxation.
Proof.Let x∗ denote the optimal solution
w · x∗ = W otherwise increase some xi
∀i ∈ B : x∗
i = 1 otherwiseincrease x∗
i and decrease some x∗
j for j ∈ {k} ∪ S∀j ∈ S : x∗
j = 0 otherwisedecrease x∗
j and increase x∗
k
This only leaves xk =W−
∑i∈B wi
wk
1015
20
20
WB
S
k
– 28. April 2010 14/44
![Page 18: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/18.jpg)
xi =
1 if i ∈ BW−
∑i∈B wi
wkif i = k
0 if i ∈ S
Lemmax is the optimal solution of the linear relaxation.
Proof.Let x∗ denote the optimal solution
w · x∗ = W otherwise increase some xi
∀i ∈ B : x∗
i = 1 otherwiseincrease x∗
i and decrease some x∗
j for j ∈ {k} ∪ S∀j ∈ S : x∗
j = 0 otherwisedecrease x∗
j and increase x∗
k
This only leaves xk =W−
∑i∈B wi
wk
1015
20
20
WB
S
k
– 28. April 2010 14/44
![Page 19: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/19.jpg)
xi =
1 if i ∈ BW−
∑i∈B wi
wkif i = k
0 if i ∈ S
Lemma
opt≤∑
i
xipi ≤ 2opt
Proof.We have
∑i∈B pi ≤ opt. Furthermore,
since wk < W , pk ≤ opt. We get
opt≤∑
i
xipi ≤∑
i∈B
pi + pk
≤ opt+ opt= 2opt
1015
20
20
WB
S
k
– 28. April 2010 15/44
![Page 20: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/20.jpg)
Two-approximation of Knapsack
xi =
1 if i ∈ BW−
∑i∈B wi
wkif i = k
0 if i ∈ S
Exercise: Prove that either B or {k} is a2-approximation of the (nonrelaxed)knapsack problem.
1015
20
20
WB
S
k
– 28. April 2010 16/44
![Page 21: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/21.jpg)
Dynamic Programming— Building it Piece By Piece
Principle of Optimality
An optimal solution can be viewed as constructed ofoptimal solutions for subproblems
Solutions with the same objective values areinterchangeable
Example: Shortest Paths
Any subpath of a shortest path is a shortest path
Shortest subpaths are interchangeable
s tu v
– 28. April 2010 17/44
![Page 22: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/22.jpg)
Dynamic Programming by Capacityfor the Knapsack Problem
DefineP(i ,C) = optimal profit from items 1,. . . ,i using capacity ≤ C.
Lemma
∀1 ≤ i ≤ n : P(i ,C) = max(P(i − 1,C),
P(i − 1,C − wi) + pi)
Of course this only holds for C large enough: we must haveC ≥ wi .
– 28. April 2010 18/44
![Page 23: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/23.jpg)
Lemma∀1 ≤ i ≤ n : P(i ,C) = max(P(i − 1,C), P(i − 1,C − wi) + pi)
Proof.To prove: P(i ,C) ≤ max(P(i − 1,C),P(i − 1,C − wi) + pi)Assume the contrary ⇒∃x that is optimal for the subproblem such that
P(i − 1,C) < p · x ∧ P(i − 1,C − wi) + pi < p · x
Case xi = 0: x is also feasible for P(i − 1,C). Hence,P(i − 1,C) ≥ p · x. Contradiction
Case xi = 1: Setting xi = 0 we get a feasible solution x′ forP(i − 1,C − wi) with profit p · x′ = p · x − pi . Addpi . . .
– 28. April 2010 19/44
![Page 24: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/24.jpg)
Computing P(i ,C) bottom up:
Procedure knapsack(p, c, n, W )array P[0 . . .W ] = [0, . . . ,0]bitarray decision[1 . . .n,0 . . .W ] = [(0, . . . ,0), . . . , (0, . . . ,0)]for i := 1 to n do
// invariant:∀C ∈ {1, . . . ,W} : P[C] = P(i − 1,C)for C := W downto wi do
if P[C − wi ] + pi > P[C] thenP[C] := P[C − wi ] + pi
decision[i ,C] := 1
– 28. April 2010 20/44
![Page 25: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/25.jpg)
Recovering a Solution
C := Warray x[1 . . . n]for i := n downto 1 do
x[i] := decision[i ,C]if x [i] = 1 then C := C − wi
endforreturn x
Analysis:
Time: O(nW ) pseudo-polynomial
Space: W +O(n) words plus Wn bits.
– 28. April 2010 21/44
![Page 26: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/26.jpg)
Example: A Knapsack Instance
maximize (10,20,15,20) · xsubject to (1,3,2,4) · x ≤ 5P(i ,C), (decision[i ,C])
i \ C 0 1 2 3 4 50 0 0 0 0 0 01 0, (0) 10, (1) 10, (1) 10, (1) 10, (1) 10, (1)234
– 28. April 2010 22/44
![Page 27: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/27.jpg)
Example: A Knapsack Instance
maximize (10,20,15,20) · xsubject to (1,3,2,4) · x ≤ 5Entries in table are P(i ,C), (decision[i ,C])i \ C 0 1 2 3 4 50 0 0 0 0 0 01 0, (0) 10, (1) 10, (1) 10, (1) 10, (1) 10, (1)234
We check each time whether P[C − wi ] + pi > P[C]
– 28. April 2010 23/44
![Page 28: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/28.jpg)
Example: A Knapsack Instance
maximize (10,20,15,20) · xsubject to (1,3,2,4) · x ≤ 5Entries in table are P(i ,C), (decision[i ,C])i \ C 0 1 2 3 4 50 0 0 0 0 0 01 0, (0) 10, (1) 10, (1) 10, (1) 10, (1) 10, (1)234
We check each time whether P[C − wi ] + pi > P[C]
– 28. April 2010 24/44
![Page 29: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/29.jpg)
Example: A Knapsack Instance
maximize (10,20,15,20) · xsubject to (1,3,2,4) · x ≤ 5Entries in table are P(i ,C), (decision[i ,C])i \ C 0 1 2 3 4 50 0 0 0 0 0 01 0, (0) 10, (1) 10, (1) 10, (1) 10, (1) 10, (1)2 0, (0) 10, (0) 10, (0) 20, (1) 30, (1) 30, (1)34
We check each time whether P[C − wi ] + pi > P[C]
– 28. April 2010 25/44
![Page 30: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/30.jpg)
Example: A Knapsack Instance
maximize (10,20,15,20) · xsubject to (1,3,2,4) · x ≤ 5Entries in table are P(i ,C), (decision[i ,C])i \ C 0 1 2 3 4 50 0 0 0 0 0 01 0, (0) 10, (1) 10, (1) 10, (1) 10, (1) 10, (1)2 0, (0) 10, (0) 10, (0) 20, (1) 30, (1) 30, (1)3 0, (0) 10, (0) 15, (1) 25, (1) 30, (0) 35, (1)4
We check each time whether P[C − wi ] + pi > P[C]
– 28. April 2010 26/44
![Page 31: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/31.jpg)
Example: A Knapsack Instance
maximize (10,20,15,20) · xsubject to (1,3,2,4) · x ≤ 5Entries in table are P(i ,C), (decision[i ,C])i \ C 0 1 2 3 4 50 0 0 0 0 0 01 0, (0) 10, (1) 10, (1) 10, (1) 10, (1) 10, (1)2 0, (0) 10, (0) 10, (0) 20, (1) 30, (1) 30, (1)3 0, (0) 10, (0) 15, (1) 25, (1) 30, (0) 35, (1)4 0, (0) 10, (0) 15, (0) 25, (0) 30, (0) 35, (0)
We check each time whether P[C − wi ] + pi > P[C]
– 28. April 2010 27/44
![Page 32: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/32.jpg)
Dynamic Programming by Profitfor the Knapsack Problem
DefineC(i ,P) = smallest capacity from items 1,. . . ,i giving profit ≥ P.
Lemma
∀1 ≤ i ≤ n : C(i ,P) = min(C(i − 1,P),
C(i − 1,P − pi) + wi)
– 28. April 2010 28/44
![Page 33: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/33.jpg)
Dynamic Programming by Profit
Let P:= bp · x∗c where x∗ is the optimal solution of the linearrelaxation.Thus P is the value (profit) of this solution.
Time: O(
nP)
pseudo-polynomial
Space: P +O(n) words plus Pn bits.
– 28. April 2010 29/44
![Page 34: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/34.jpg)
A Faster Algorithm
Dynamic programs are only pseudo-polynomial-timeA polynomial-time solution is not possible (unless P=NP...),because this problem is NP-hardHowever, it would be possible if the numbers in the input weresmall (i.e. polynomial in n)To get a good approximation in polynomial time, we are goingto ignore the least significant bits in the input
– 28. April 2010 30/44
![Page 35: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/35.jpg)
Fully Polynomial Time ApproximationScheme
Algorithm A is a(Fully) Polynomial Time Approximation Scheme
forminimizationmaximization
problem Π if:
Input: Instance I, error parameter ε
Output Quality: f (x)≤
≥(1+ε
1−ε)opt
Time: Polynomial in |I| (and 1/ε)
– 28. April 2010 31/44
![Page 36: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/36.jpg)
Example Bounds
PTAS FPTAS
n + 21/ε n2 +1ε
nlog 1ε n +
1ε4
n1ε n/ε
n42/ε3 ...
n + 221000/ε ......
...
– 28. April 2010 32/44
![Page 37: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/37.jpg)
Problem classes
We can classify problems according to the approximation ratioswhich they allow.
APX: constant approximation ratio achievable in timepolynomial in n (Metric TSP, Vertex Cover)
PTAS: 1 + ε achievable in time polynomial in n for anyε > 0 (Euclidean TSP)
FPTAS: 1+ achievable in time polynomial in n and 1/ε forany > 0 (Knapsack)
– 28. April 2010 33/44
![Page 38: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/38.jpg)
FPTAS → optimal solution
By choosing ε small enough, you can guarantee that thesolution you find is in fact optimal. The running time will dependon the size of the optimal solution, and will thus again not bestrictly polynomial-time (for all inputs).
– 28. April 2010 34/44
![Page 39: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/39.jpg)
FPTAS for Knapsack
Recall that pi ∈ N for all i !
P:= maxi pi // maximum profit
K :=εPn
// scaling factor
p′
i :=⌊pi
K
⌋// scale profits
x′:= dynamicProgrammingByProfit(p′, c,C)output x′
– 28. April 2010 35/44
![Page 40: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/40.jpg)
FPTAS for Knapsack
Recall that pi ∈ N for all i !
P:= maxi pi // maximum profit
K :=εPn
// scaling factor
p′
i :=⌊pi
K
⌋// scale profits
x′:= dynamicProgrammingByProfit(p′, c,C)output x′
Example:ε = 1/3,n = 4,P = 20 → K = 5/3p = (11,20,16,21) → p′ = (6,12,9,12) (or p′ = (2, 4, 3, 4))
W
1121
1620
– 28. April 2010 36/44
![Page 41: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/41.jpg)
Lemmap · x′ ≥ (1 − ε)opt.
Proof.Consider the optimal solution x∗.
p · x∗ − K p′ · x∗ =∑
i∈x∗
(pi − K
⌊pi
K
⌋)
≤∑
i∈x∗
(pi − K
(pi
K− 1))
= |x∗|K≤ nK ,
i.e., K p′ · x∗ ≥ p · x∗ − nK . Furthermore,
K p′ · x∗ ≤ K p′ · x′ =∑
i∈x′
K⌊pi
K
⌋≤∑
i∈x′
Kpi
K= p · x′.
We use that x′ is an optimal solution for the modified problem.
p · x′ ≥K p′ · x∗ ≥ p · x∗ − nK = opt− εP ≥ (1 − ε)opt– 28. April 2010 37/44
![Page 42: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/42.jpg)
Lemmap · x′ ≥ (1 − ε)opt.
Proof.Consider the optimal solution x∗.
p · x∗ − K p′ · x∗ =∑
i∈x∗
(pi − K
⌊pi
K
⌋)
≤∑
i∈x∗
(pi − K
(pi
K− 1))
= |x∗|K≤ nK ,
i.e., K p′ · x∗ ≥ p · x∗ − nK . Furthermore,
K p′ · x∗ ≤ K p′ · x′ =∑
i∈x′
K⌊pi
K
⌋≤∑
i∈x′
Kpi
K= p · x′.
We use that x′ is an optimal solution for the modified problem.
p · x′ ≥K p′ · x∗ ≥ p · x∗ − nK = opt− εP ≥ (1 − ε)opt– 28. April 2010 37/44
![Page 43: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/43.jpg)
Lemmap · x′ ≥ (1 − ε)opt.
Proof.Consider the optimal solution x∗.
p · x∗ − K p′ · x∗ ≤∑
i∈x∗
(pi − K
(pi
K− 1))
= |x∗|K≤ nK ,
i.e., K p′ · x∗ ≥ p · x∗ − nK . Furthermore,
K p′ · x∗ ≤ K p′ · x′ =∑
i∈x′
K⌊pi
K
⌋≤∑
i∈x′
Kpi
K= p · x′.
Hence,
p · x′ ≥K p′ · x∗ ≥ p · x∗ − nK = opt− εP ≥ (1 − ε)opt
– 28. April 2010 38/44
![Page 44: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/44.jpg)
Lemma
Running time O(n3/ε
).
Proof.The running time of dynamic programming dominates.
Recall that this is O(
nP ′
)where P ′ = bp′ · x∗c.
We have
nP ′ ≤n · (n · max p′
i) = n2⌊
PK
⌋= n2
⌊PnεP
⌋≤
n3
ε.
– 28. April 2010 39/44
![Page 45: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/45.jpg)
A Faster FPTAS for Knapsack
Simplifying assumptions:
1/ε ∈ N: Otherwise ε:= 1/ d1/εe.
Upper bound P is known: Use linear relaxation to get a quick2-approximation.
mini pi ≥ εP: Treat small profits separately. For these itemsgreedy works well. (Costs a factor O(log(1/ε))time.)
– 28. April 2010 40/44
![Page 46: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/46.jpg)
A Faster FPTAS for Knapsack
M:=1ε2 ; K := Pε2 = P/M
p′
i :=⌊pi
K
⌋// p′
i ∈{1ε , . . . ,M
}
value of optimal solution was at most P, is now MDefine buckets Cj :=
{i ∈ 1..n : p′
i = j}
keep only the⌊
Mj
⌋lightest (smallest) items from each Cj
do dynamic programming on the remaining items
Lemmapx ′ ≥ (1 − ε)opt.
Proof.Similar as before, note that |x| ≤ 1/ε for any solution.
– 28. April 2010 41/44
![Page 47: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/47.jpg)
LemmaRunning time O(n + Poly(1/ε)).
Proof.
preprocessing time: O(n)
values: M = 1/ε2
pieces:M∑
i=1/ε
⌊Mj
⌋≤ M
M∑
i=1/ε
1j≤ M ln M = O
(log(1/ε)
ε2
)
time dynamic programming: O(
log(1/ε)ε4
)
– 28. April 2010 42/44
![Page 48: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/48.jpg)
The Best Known FPTAS
[Kellerer, Pferschy 04]
O
(min
{n log
1ε+
log2 1ε
ε3 , . . .
})
Less buckets Cj (nonuniform)
Sophisticated dynamic programming
– 28. April 2010 43/44
![Page 49: The Knapsack Problem 20 - KIT · Example: The Knapsack Problem maximize p ·x subject to w ·x ≤ W,xi ∈ {0,1} for 1 ≤ i ≤ n. xi = 1 iff item i is put into the knapsack. 0/1](https://reader034.fdocuments.in/reader034/viewer/2022051810/601695c374193e64b50271b0/html5/thumbnails/49.jpg)
Optimal Algorithm for the KnapsackProblem
The best work in near linear time for almost all inputs! Both in aprobabilistic and in a practical sense.[Beier, Vocking, An Experimental Study of Random KnapsackProblems, European Symposium on Algorithms, 2004.][Kellerer, Pferschy, Pisinger, Knapsack Problems, Springer2004.]Main additional tricks:
reduce to core items with good profit density,
Horowitz-Sahni decomposition for dynamic programming
– 28. April 2010 44/44