01 knapsack using backtracking

Backtracking Technique

Eg. Big Castle – Large Rooms & “Sleeping Beauty”

• Systematic search - BFS, DFS• Many paths led to nothing but “dead-ends”

• Can we avoid these unnecessary labor? (bounding function or promising function)

• DFS with bounding function (promising function)• Worst-Case : Exhaustive Search

Backtracking Technique

• Useful because it is efficient for “many” large instances

• NP-Complete problems

Eg. 0-1 knapsack problem D.P. : O(min(2n, nW)) Backtracking : can be very efficient if good bounding function(promising function)

• Recursion

Outline of Backtracking Approach

• Backtracking : DFS of a tree except that nodes are visited if promising

• DFS(Depth First Search)

• Procedure d_f_s_tree(v: node)var u : node{ visit v; // some action // for each child u of v do d_f_s_tree(u);}

1

2

3 5 7 114

6

8 9 10

N-Queens Problem

N-Queens Problem

• nⅹn Chess board• n-Queens Eg. 8-Queens problem

, ,)(2

2

n

nn n ,! , nnn

QQ

QQ

1 2 3 4 5 6 7 8

1 2 3 4 5 6 7 8

N-Queens Problem: DFS

• State Space Tree for 4-Queens Problem

N-Queens Problem: DFS (Same Col/Row x)


X1=1

X2=2 3 4

43

4 3 4 2 3 2 3 2

2 4 32 3 4 1 4 1 3 2 4

1 3 4 1 2 4 321

2 3 4

(x1, x2, x3, x4)=(2, 4, 1, 3)

• #leaf nodes : n!=4!• DFS : “Backtrack” at dead ends – 4! leaf nodes (n!)Cf. Backtracking : “Backtrack” if non-promising

(pruning)

N-Queens Problem: Backtracking

• Promising Function(Bounding Function) (i) same row ⅹ (ii) same column ⅹ (col(i) ≠ col(k)) (iii) same diagonal ⅹ (|col(i)-col(k)|≠|i-k|)

Q (i,

col(i)) Q (k,

col(k))

Q (i,col(i))

Q (k,col(k))

N-Queens Problem: Backtracking


N-Queens Problem

• Better (faster) AlgorithmMonte Carlo Algorithm“place almost all queens randomly, then do remaining queens using backtracking.”

#Nodes CheckedDFS (nn)

#Nodes CheckedDFS

(same col/row X)(n!)

#Nodes CheckedBacktracking

34119,173,9619.73ⅹ1012

1.20ⅹ1016

2440,320

4.79ⅹ108

8.72ⅹ1010

6115,721

1.01ⅹ107

3.78ⅹ108

48

1214

n

Graph Coloring

• M-coloring problem:Color undirected graph with ≤m colors2 adjacent vertices : different colors

(Eg)V1 V2

V4 V3

2-coloring X

3-coloring O

Graph Coloring

• State-Space Tree : mn leaves• Promising function : check adjacent vertices for the same color

m

start

1 2 3

21 3

21 3

21 3

X

X

X

X X

Hamiltonian Circuits Problem

• TSP problem in chapter 3.– Brute-force: n!, (n-1)!– D.P. : – When n=20, B.F. 3,800 years D.P. 45 sec.– More efficient solution? See chapter 6

• Given an undirected or directed graph, find a tour(HC). (need not be S.P.)

32)2)(1( nnn

Hamiltonian Circuits Problem

• State-Space Tree(n-1)! Leaves : worst-case

• Promising function:1. i-th node (i+1)-th node2.(n-1)-th node 0-th node3. i-th node ≠ 0 ~ (i-1)-th node

V1 V2

V6V5

V3

V7

V4

V8

V1 V2

V5

V3

V4

HC : X

(Eg)

Sum-of-Subsets Problem

• A special case of 0/1-knapsack• S = {item1, item2,…, itemn}, w[1..n] = (w1, w2, …, wn) weights & W Find a subset A S s.t. ∑w⊆ i=W

(Eg) n=3, w[1..3]=(2,4,5), W=6 Solution : {w1,w2}

• NP-Complete• State Space Tree : 2n leaves

A

w1

w2

w3

w2

w3 w3 w3

O

O O

OOOO

{w1,w2}

Sum-of-Subsets Problem

• Assumption : weights are in sorted order• Promising function

ⅹ weight(up_to_that_node) + w i+1 > W (at i-th level) ⅹ weight(up_to_that_node) + total(remaining) < W(Eg) n=4, (w1,w2,w3,w4)=(3,4,5,6), W=13

ⅹ

12

7

3 97 8

13

4

0437

0

03

3 0

4

5

0 04

55

6

0 00

0ⅹ

ⅹ:

0+5+6=11<13

ⅹⅹ:

9+6=15>13

ⅹⅹ

w1=3

w2=4

w3=5

w4=6

0-1 Knapsack Problem

• w[1..n] = (w1, w2,…, wn) p[1..n] = (p1, p2,…, pn) W: Knapsack size• Determine A S maximizing⊆

• State-Space Tree

• 2n leaves

x1=1x2=

1x3=1

x4=1

00

0

00000

0000

001

11 1 1 0

11 1

11

x2=1

WwtspA

iA

i ..

Branch-and-Bound Backtracking

Non-optimization P. n-Queens, m-coloring…

Optimization Prob. 0/1 Knapsack

Compute promising fcn.

Branch & Bound Optimization Prob.- compute promising fcn.

Maximization Prob. → upper bound in each node

Minimization Prob. → lower bound in each node

BFS with B&B

Best-First-Search with B&B

Breadth First Search

procedure BFS(T : tree);{ initialize(Q);

v = root of T;visit v;enqueue(Q,v);while not empty(Q) do {

dequeue(Q,v);for each child u of v do { visit u;

enqueue(Q,u); } }}

1

2 3

4 5 6 7

8 9

1

2

5 6 7 8 9

3 4

10 11

12 13 14 15

BFS of a graph

BFS of a tree


• BFS with B&B pruning (Eg) n=4, p[1…4]=(40,30,50,10), w[1…4]=(2,5,10,5),

W=1600

115

402

115

00

82

707

115

402

98

120170

707

80

801280

707

70

901298

402

50

100170

901290

305

82

801582

305

40

00

60

p1=40, w1=2

p2=30, w2=5

p3=50, w3=10

p4=10, w4=5


– Bound = current profit + profit of remaining “fractional” K.S.

– Non-promising if bound ≤ max profit (or weight ≥W)(Eg) n=4, p[1…4]=(40,30,50,10), w[1…4]=(2,5,10,5), W=160

0115

402

115

00

82

707

115

402

98

120170

707

80

801280

707

70

901298

402

50

100170

901290

305

82

801582

305

40

00

60

p1=40, w1=2

p2=30, w2=5

p3=50, w3=10

p4=10, w4=5


• Best First Search with B&B pruning

00

115

402

115

00

82

707

115

402

98

120170

707

80

901298

402

50

100170

p1=40, w1=2

p2=30, w2=5

p3=50, w3=10

p4=10, w4=5 901290

01 knapsack using backtracking

Technology

Transcript of 01 knapsack using backtracking