Backtracking

© Jeff Parker, 2009

Cogito, ergo spud: I think, therefore I yam.

2

Outline

Backtracking

Problem - 8 Queens

Demonstration of 4 Queens

Code for 8 Queens problem

How much will this cost?

A better solution

Turnpike Problem

3

Backtracking – the 8 Queens

Problem: place 8 queens on a chess board so that none attack each other

General class of problems can be solved by backtracking

4

Decreases size of search space

If we were to examine all placements of queens, would take C(64,8)64*63*62*61*60*59*58*57*568 * 7 * 6 * 5 * 4 * 3 * 2 * 1 = 4,426,165,368

Restrict queen i to col i means we only look at 8^8 positions8 * 8 * 8 * 8 * 8 * 8 * 8 * 8 = (2^3) ^ 8 = 2 ^ 24 ~ 16,000,000

Restrict queen i to col i and avoid rows in use gives8 * 7 * 6 * 5 * 4 * 3 * 2 * 1 = 40,320possible moves

Backtracking can reduce this far more.When we place the first queen we eliminate one of the 7 second slots, so we don't

need to consider 2 * 6 * 5 * 4 * 3 * 2 * 1 = 1440 extensionsIn fact, for an 8x8 board we look at 114 partial boards

The only complete boards we look at are solutions....

5

Backtracking Demo

for (r0 = 0; r0 < 4; r0++)

if safe(r0, 0) {

place queen at (r0, 0)

for (r1 = 0; r1 < 4; r1++)

if safe(r1, 1) {

place queen at (r1, 1)

for (r2 = 0; r2 < 4; r2++)

In the slides to follow, read from left to right, top to bottom.

6

7

8

A solution to the 4 queens problem

9

Stack Tracing

In effect, we are placing our partial solution on the system's procedure stack.

We extend in a new stack frame.

If we fail, we remove frame, and return to our position

If we find a safe row, we try to extend.

If no row works out, we pop the stack and retry in the previous column

Recursion hides the stack - simply the procedure stack

10

Backtracking Skeleton// Not every backtracking example has all points below: useful templateboolean backtracking(some parameters) {

if (we have a solution) report results and return true;if (there is no hope) return false;for (first position to last postion) {

if (this looks like a legal postion) {Record position// Can I use this step and solve the rest of the problem?if (backtracking(parameters modified to reflect my new position))

// Success! Record position and returnremember my positionreturn true;

elseremove any traces from this call and try next iteration of

loopreturn false; // If I reach here, I was not successful. Backtrack

11

Backtracking for 8 Queensdef check(lst, col, rowfree, upfree, downfree):

"""Try to extend a solution into col."""print col, lst # Debugging: 3 [0, 3, 1, -1]

if (col == len(lst)):print "Success!"printBoard(lst)return True

for row in xrange(len(lst)):if (safe(lst, row, col, rowfree, upfree, downfree)):

place(lst, row, col, rowfree, upfree, downfree)if (check(lst, col+1, rowfree, upfree, downfree)):

return Trueremove(lst, row, col, rowfree, upfree, downfree)

return False # Backtrack

12

Why use Recursion?Why not use nested for loops, as suggested?for (r0 = 0; r0 < 4; r0++)

if safe (r0, 0) {place queen at (r0, 0) for (r1 = 0; r1 < 4; r1++)

if safe (r1, 1) {place queen at (r1, 1) for (r2 = 0; r2 < 4; r2++)...

• Does not scale to different sized boards• You must duplicate identical code (place and remove). An error in

one spot is hard to find

13

4 Queens Results0 [-1, -1, -1, -1]1 [0, -1, -1, -1]2 [0, 2, -1, -1]Backtrack2 [0, 3, -1, -1]3 [0, 3, 1, -1]BacktrackBacktrackBacktrack1 [1, -1, -1, -1]2 [1, 3, -1, -1]3 [1, 3, 0, -1]4 [1, 3, 0, 2]Success! . * .* . . *. * .

14

3 Queens Results0 [-1, -1, -1]1 [0, -1, -1]2 [0, 2, -1]BacktrackBacktrack1 [1, -1, -1]Backtrack1 [2, -1, -1]2 [2, 0, -1]BacktrackBacktrackBacktrackCould not solve for a board with side 3

15

8 Queens Results* . . . .. . . * . . * . .. . . . * * . . .. . * . . . . * .. * . .

16

Finding All Solutions// No longer returns after success: keep tryingvoid backtracking(some parameters) {

if (we have a solution) report the position and return;if (there is no hope) return;for (first position to last postion) {

if (this looks like a legal postion) {# if (backtracking(parameters modified))

# return true;backtracking(parameters modified)

remove traces of this call // . . . and try the next iteration

17

Find all Solutionsdef check(lst, col, rowfree, upfree, downfree):

"""Find All Solutions"""

if (col == len(lst)):

print "Success!"

printBoard(lst)

return True

for row in xrange(len(lst)):

if (safe(lst, row, col, rowfree, upfree, downfree)):

place(lst, row, col, rowfree, upfree, downfree)

#if (check(lst, col+1, rowfree, upfree, downfree)):

# return True

check(lst, col+1, rowfree, upfree, downfree)

remove(lst, row, col, rowfree, upfree, downfree)

return False # Backtrack

18

Data StructuresHow do we store the board?

We could use a two dimensional array of booleans.void place(int row, int column){

board[row][column] = QUEEN;}

Operations needed for backtrackingCheck to see if queen is safe - safePlace a queen - placeRemove Queen - remove

Takes a fixed amount of time to set or remove.But which of the three operations above do we do most often?

if (safe(...)):place(...)if (check(...)):

return Trueremove(...)

19

Python Profiler counts the calls2166 function calls (2053 primitive calls) in 0.045 CPU seconds

Ordered by: standard name

ncalls tottime percall cumtime percall filename:lineno(function) 38 0.000 0.000 0.000 0.000 :0(insert) 696 0.006 0.000 0.006 0.000 :0(len) 1 0.001 0.001 0.001 0.001 :0(setprofile) 1 0.000 0.000 0.044 0.044 <string>:1(<module>) 876 0.009 0.000 0.011 0.000 Queens.py:19(safe) 113 0.002 0.000 0.003 0.000 Queens.py:29(place) 105 0.002 0.000 0.003 0.000 Queens.py:36(remove) 219 0.003 0.000 0.003 0.000 Queens.py:43(prettyPrint) 114/1 0.021 0.000 0.043 0.043 Queens.py:49(check) 1 0.000 0.000 0.000 0.000 Queens.py:7(printBoard) 1 0.000 0.000 0.043 0.043 Queens.py:81(solve) 0 0.000 0.000 profile:0(profiler) 1 0.000 0.000 0.045 0.045 profile:0(solve([-1, -1, -1, -1, -1, -1, -1, -1]))

20

Testing a new position

When we wish to test position (i, j), we attack over 20 squares.

We don't need to test this col, nor the spots to our right. Worst case is 14 squares

(x, j), (i, y), (i+x, j+x), (i+x, j-x)

Still, there is a good deal of work to be done here....

21

Alternative

Store board as an array of row positions. The board above is

[5, 3, 0, 4, 1,.....]

Nothing new yet...

Store an array of booleans rowFreeIs this row free of queens?

22

Smarter Storage

Store 3 arrays of boolean

rowfree, upfree, downfree

Is this row under attack?

Is this diagonal under attack? ...

Placement & removal is now more expensive: must touch 4 arrays

However, when testing we only examine 3 array locations (col is implicit)

23

Is this spot safe?

# When testing we only examine 3 array locations (col is implicit)

def safe(lst, row, col, rowfree, upfree, downfree):"""Is it safe to place a queen at (row, col)?"""if (not rowfree[row]):

return Falseif (not upfree[row+col]):

return Falseif (not downfree[row-col+len(lst)-1]):

return Falsereturn True

(0, 0)(0, 1)

(0, 2)(0, 3)

(1, 0)(1, 1)

(1, 2)(1, 3)

(2, 0)(2, 1)

(2, 2)(2, 3)

(3, 0)(3, 1)

(3, 2)(3, 3)

24

Place and Remove(0, 0)

(0, 1)(0, 2)

(0, 3)

(1, 0)(1, 1)

(1, 2)(1, 3)

(2, 0)(2, 1)

(2, 2)(2, 3)

(3, 0)(3, 1)

(3, 2)(3, 3)

(0, 0)(0, 1)

(0, 2)(0, 3)

(1, 0)(1, 1)

(1, 2)(1, 3)

(2, 0)(2, 1)

(2, 2)(2, 3)

(3, 0)(3, 1)

(3, 2)(3, 3)

def place (lst, row, col, rowfree, upfree, downfree):"""Put a queeen in pos (row, col)."""lst[col] = rowrowfree[row] = Falseupfree[row+col] = Falsedownfree[row-col+len(lst)-1] = False

def remove(lst, row, col, rowfree, upfree, downfree):"""Remove a queeen from pos (row, col)."""lst[col] = -1rowfree[row] = Trueupfree[row+col] = Truedownfree[row-col+len(lst)-1] = True

25

Initialize the arrays

def solve(lst):"""Set things up for a run."""rowfree = [ ]upfree = [ ]downfree = [ ]for x in xrange(len(lst)):

rowfree.insert(x, True)

for x in xrange(2*len(lst)- 1):upfree.insert(x, True)downfree.insert(x, True)

if (not check(lst, 0, rowfree, upfree, downfree, 0)):print "Can not solve for a board with side", len(lst)

solve([-1, -1, -1, -1])

(0, 0)(0, 1)

(0, 2)(0, 3)

(1, 0)(1, 1)

(1, 2)(1, 3)

(2, 0)(2, 1)

(2, 2)(2, 3)

(3, 0)(3, 1)

(3, 2)(3, 3)

26

Profile the run

import profile...profile.run('solve([-1, -1, -1, -1, -1, -1, -1, -1])')

# ================ Output =================

2166 function calls (2053 primitive calls) in 0.045 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 38 0.000 0.000 0.000 0.000 :0(insert) 696 0.006 0.000 0.006 0.000 :0(len) 1 0.001 0.001 0.001 0.001 :0(setprofile) 1 0.000 0.000 0.044 0.044 <string>:1(<module>) 876 0.009 0.000 0.011 0.000 Queens.py:19(safe) 113 0.002 0.000 0.003 0.000 Queens.py:29(place) 105 0.002 0.000 0.003 0.000 Queens.py:36(remove)

27

SummaryBacktracking allows us to search a large solution space

It organizes the search, and avoids wasting time

Clever choice of data structure can save a great deal of time

Rather than looking at a dozen squares in 8 Queens, we could test 3 locations

Clever modification of algorithms can yield large savings

Representation should be tuned to the requirements

28

Turnpike

29

Sample Run

Extend [0] [] vs [2, 8, 10]Try extending by adding 2 to get [0, 2]

Extend [0, 2] [2] vs [2, 8, 10]Try extending by adding 8 to get [0, 2, 8]Try extending by adding 10 to get [0, 2, 10]

Extend [0, 2, 10] [2, 8, 10] vs [2, 8, 10]Success!! [0, 2, 10] [2, 8, 10]

30

Turnpike

31

Utilitydef insrt(lst, itm):

"""Insert item itm into sorted list lst"""spot = -1for x in xrange(len(lst)):

if (lst[x] < itm):spot = x

else:break

lst.insert(spot+1, itm)return lst

def differences(a):"""Take {0, 2, 4, 7, 10} generate {2, 2, 3, 3, 4, 5, 6, 7, 8, 10}"""

lst = []for x in xrange(len(a)):

for y in xrange(x+1, len(a)):insrt(lst, a[y] - a[x])

return lst

32

Subset

def subset(a, b):"""Assume a and b are sorted. Is a a subset of b?"""if (len(a) > len(b)):

return Falseif (0 == len(a)):

return Truex = 0for y in xrange(len(b)):

if (a[x] < b[y]):return False

if (a[x] == b[y]):x = x + 1if (x == len(a)):

return Truereturn False

33

Pretty Printdef prettyPrint(depth, string):

"""Indent to show the levels of recursion"""for lp in xrange(depth):

print "\t",print string,

def extend(cand, lst, depth):"""Cand generates a subset of list. Try to add an elt."""prettyPrint(depth, "Extend")print cand, differences(cand), "vs", lst...

if (extend(nwLst, lst, depth+1)):

Extend [0] [] vs [2, 8, 10]Try extending by adding 2 to get [0, 2]

Extend [0, 2] [2] vs [2, 8, 10]Try extending by adding 8 to get [0, 2, 8]Try extending by adding 10 to get [0, 2, 10]

Extend [0, 2, 10] [2, 8, 10] vs [2, 8, 10]Success!! [0, 2, 10] [2, 8, 10]

34

ImprovementsExtend [0] [] vs [2, 2, 2, 3, 3, 4, 5, 5, 6, 7, 8, 8, 10, 10, 12]Try extending by adding 2 to get [0, 2]

Extend [0, 2] [2] vs [2, 2, 2, 3, 3, 4, 5, 5, 6, 7, 8, 8, 10, 10, 12]Try extending by adding 3 to get [0, 2, 3]Try extending by adding 3 to get [0, 2, 3]Try extending by adding 4 to get [0, 2, 4]

Extend [0, 2, 4] [2, 2, 4] vs [2, 2, 2, 3, 3, 4, 5, 5, 6, 7, 8, 8, 10, 10, 12]Try extending by adding 3 to get [0, 2, 3, 4]Try extending by adding 3 to get [0, 2, 3, 4]Try extending by adding 5 to get [0, 2, 4, 5]Try extending by adding 5 to get [0, 2, 4, 5]Try extending by adding 6 to get [0, 2, 4, 6]Try extending by adding 7 to get [0, 2, 4, 7]

Extend [0, 2, 4, 7] [2, 2, 3, 4, 5, 7] vs [2, 2, 2, 3, 3, 4, 5, 5, 6, 7,...Try extending by adding 3 to get [0, 2, 3, 4, 7]Try extending by adding 3 to get [0, 2, 3, 4, 7]Try extending by adding 5 to get [0, 2, 4, 5, 7]Try extending by adding 5 to get [0, 2, 4, 5, 7]Try extending by adding 6 to get [0, 2, 4, 6, 7]Try extending by adding 8 to get [0, 2, 4, 7, 8]Try extending by adding 8 to get [0, 2, 4, 7, 8]Try extending by adding 10 to get [0, 2, 4, 7, 10]

35

ImprovementsExtend [0] [] vs [2, 2, 2, 3, 3, 4, 5, 5, 6, 7, 8, 8, 10, 10, 12]Try extending by adding 2 to get [0, 2]

Extend [0, 2] [2] vs [2, 2, 2, 3, 3, 4, 5, 5, 6, 7, 8, 8, 10, 10, 12]Try extending by adding 3 to get [0, 2, 3]Try extending by adding 3 to get [0, 2, 3]Try extending by adding 4 to get [0, 2, 4]

Extend [0, 2, 4] [2, 2, 4] vs [2, 2, 2, 3, 3, 4, 5, 5, 6, 7, 8, 8, 10, 10, 12]Try extending by adding 3 to get [0, 2, 3, 4]Try extending by adding 3 to get [0, 2, 3, 4]Try extending by adding 5 to get [0, 2, 4, 5]Try extending by adding 5 to get [0, 2, 4, 5]Try extending by adding 6 to get [0, 2, 4, 6]Try extending by adding 7 to get [0, 2, 4, 7]

Extend [0, 2, 4, 7] [2, 2, 3, 4, 5, 7] vs [2, 2, 2, 3, 3, 4, 5, 5, 6, 7,...Try extending by adding 3 to get [0, 2, 3, 4, 7]Try extending by adding 3 to get [0, 2, 3, 4, 7]Try extending by adding 5 to get [0, 2, 4, 5, 7]Try extending by adding 5 to get [0, 2, 4, 5, 7]Try extending by adding 6 to get [0, 2, 4, 6, 7]Try extending by adding 8 to get [0, 2, 4, 7, 8]Try extending by adding 8 to get [0, 2, 4, 7, 8]Try extending by adding 10 to get [0, 2, 4, 7, 10]

36

Profile Ordered by: standard name

ncalls tottime percall cumtime percall filename:lineno(function) 342 0.003 0.000 0.003 0.000 :0(insert) 717 0.006 0.000 0.006 0.000 :0(len) 1 0.001 0.001 0.001 0.001 :0(setprofile) 1 0.000 0.000 0.027 0.027 <string>:1(<module>) 38 0.005 0.000 0.020 0.001 Turnpike.py:18(differences) 32 0.002 0.000 0.003 0.000 Turnpike.py:26(subset) 32 0.000 0.000 0.000 0.000 Turnpike.py:42(prettyPrint) 6/1 0.002 0.000 0.027 0.027 Turnpike.py:48(extend) 342 0.009 0.000 0.014 0.000 Turnpike.py:7(insrt) 1 0.000 0.000 0.029 0.029 profile:0(extend([0], [2, 2, 2, 3, 3, 4, 5, 5, 6, 7, 8, 8, 10,

10, 12], 0)) 0 0.000 0.000 profile:0(profiler)