Lecture 4:

Divide-and-Conquer

Steps in Divide and Conquer

As its name implies divide-and-conquer involves dividing a problem into smaller problems that can be more easily solved. While the specifics vary from one application to another, divide-and-conquer always includes the following three steps in some form:

Divide - Typically this steps involves splitting one problem into two problems of approximately 1/2 the size of the original problem.

Conquer - The divide step is repeated (usually recursively) until individual problem sizes are small enough to be solved (conquered) directly.

Recombine - The solution to the original problem is obtained by combining all the solutions to the sub-problems.

Divide and Conquer is not applicable to every problem class. Even when D&C works it may not provide for an efficient solution.

Binary Search - Iterative Version

procedure bin1(n:integer,x:keytype,S:list,loc:index)is lo,hi,mid : index;begin lo:=1; hi:=n; loc:=0; while lo<=hi and loc=0 loop mid:=(lo+hi)/2; if x=S(mid) then loc:=mid; elsif x<S(mid) then hi:=mid-1; else lo:=mid+1; end if; end loop;end bin1;

S

0 10

1 12

2 15

3 19

4 19

5 24

6 39

7 45

8 53

9 77

x=42lo=1 hi=9mid=4 S(4)=19

lo=5 hi=9mid=7 S(7)=45

lo=5 hi=6mid=5 S(5)=24

lo=6 hi=6mid=6 S(6)=39

lo=7 hi=6T(n) = C + lg2n D -> O(lg2n)

Binary Search - Recursive Version

function location(lo,hi : index) return index isbegin if lo>hi then return 0; else mid:=(lo+hi)/2; if x=S(mid) then return mid; elsif x<S(mid) then return location(lo,mid-1); else return location(mid+1,hi); end if; end if;

end location;

S

0 10

1 12

2 15

3 19

4 19

5 24

6 39

7 45

8 53

9 77

x=42lo=0 hi=9mid=4 S(4)=19

lo=5 hi=9mid=7 S(7)=45

lo=5 hi=6mid=5 S(5)=24

lo=6 hi=6mid=6 S(6)=39

lo=7 hi=6return 0

Analysis of Function location( )

C n<=1 T(n) =

T(n/2) + D otherwise

T(n/2)=T(n/4) + D

T(n)=(T(n/4) + D) + D

T(n)=T(n/4) + 2D

T(n)=T(n/8) + 3D

:

T(n)=T(n/2k) + kD

T(n)=T(1) + (lg2n)D

T(n)=C + (lg2n)D

T(n) -> O(lg2n)

We need a closed form expression for T(n) that does not contain a term involving T.

We assume that n>1 and note that we need to replace T(n/2) with an explicit function of n.

We can use the recurrence relation to express the term T(n/2) in another form that can, in turn be substituted back into our expression for T(n). Eventually we can write a general expression for the kth substitution.

Then we can determine a value for k (in terms of n) that sets the parameter of T equal to one. This lets us substitute T(1)=C in our recurrence and replace any other occurrences of k with terms involving n.

We can then determine the order of complexity.

Mergesort

procedure msort(n:integer; S:list) is h: constant :=n/2; m: constant :=n-h; U,V : list;begin if n>1 then U(1..h):=S(1..h); V(1..m):=S(h+1..n); msort(h,U); msort(m,V); merge(h,m,U,V,S); end if;end msort;

procedure merge(h,m:integer; U,V,S:list);i,j,k : integer;begin i:=1; j:=1; k:=1; while i<=h and j<=m loop if U(i)<V(j) then S(k):=U(i); i:=i+1; else S(k):=V(j); j:=j+1; end if; k:=k+1; end loop; if i>h then S(k..h+m):=V(j..m); else S(k..h+m):=U(i..h); end if;end merge;

Write the recurrence relation for the complexity of msort and then solve it.

Quicksort

procedure quicksort(lo,hi: integer) is

pivot : integer;

begin

if hi>lo then

partition(lo,hi,pivot);

quicksort(lo,pivot-1);

quicksort(pivot,+1,hi);

end if;

end quicksort;

procedure partition(lo,hi,pivot)is j,item : integer;begin item := S(lo); j:= lo; for i in lo+1..hi loop if S(i)<item then j:=j+1; swap(S(i),S(j)); end if; end loop; pivot:=j swap(S(lo),S(pivot));end partition;

Where is the recombine step in the quicksort divide-and-conquer algorithm?

Quicksort Example

i j 1 2 3 4 5 6 7 8 9 10 11 12 13 14

2 1 6 9 8 5 7 4 6 3 5 4 7 6 0 5

4 2 6 5 8 9 7 4 6 3 5 4 7 6 0 5

6 3 6 5 4 9 7 8 6 3 5 4 7 6 0 5

8 4 6 5 4 3 7 8 6 9 5 4 7 6 0 5

9 5 6 5 4 3 5 8 6 9 7 4 7 6 0 5

10 6 6 5 4 3 5 4 6 9 7 8 7 6 0 5

13 7 6 5 4 3 5 4 0 9 7 8 7 6 6 5

14 8 6 5 4 3 5 4 0 5 7 8 7 6 6 9

8 5 5 4 3 5 4 0 6 7 8 7 6 6 6

pivot value

items being swapped

new sublists for next pass of quicksort

Quicksort: Worst-Case PerformanceWhat does it mean?

An analysis shows that the worst-case performance for quicksort( ) is order O(n2). However, quicksort performs much better than this in practice. Does this mean that algorithm analysis is a waste of effort?

Not really. In addition to showing worst-case performance, algorithm analysis provides us with a better understanding of the conditions under which an algorithm performs well.

We can see that the quicksort algorithm performs worst on a sorted list and performs best on a randomly distributed list (at least one in which the pivot value is always near the median value of the sublist being partitioned).

Algorithm analysis helps us determine which algorithm is best for a particular application, as well as suggesting ways to modify an algorithm to improve its performance.

Can you think of a simple way to improve quicksort( )?

Searching and sorting problems are obvious candidates for D&C but the Divide-and-Conquer problem-solving method gets used in other ways you might not expect.

Other Applications for D&C

Game Strategy

Computing numeric roots

Fast Exponentiation

Suppose that we need to compute the value of an for some reasonably large n. Such problems occur in primality testing for cryptography. The simplest algorithm performs n − 1 multiplications, by computing a × a × . . . × a.

However, we can do better by observing that n = ⌊n/2⌋ + ⌈n/2⌉. If n is even, then an = (an/2)2. If n is odd, then an = a(a⌊n/2⌋)2. In either case, we have halved the size of our exponent at the cost of at most two multiplications, so O(lg2n) multiplications suffice to compute the final value.

function power(a, n) if (n = 0)

return(1) x = power(a, ⌊n/2⌋) if (n is even)

return(x2) else

return(a * x2)

Modular Exponentiation

Introduction to Algorithms - Steven Skiena

The square root of n is the number r such that r2 = n. Square root computations are performed inside every pocket calculator – but how?

Observe that the square root of n ≥ 1 must be at least 1 and at most n. Let l = 1 and r = n. Consider the midpoint of this interval, m = (l + r)/2. How does m2 compare to n?

If n ≥ m2, then the square root must be greater than m, so the algorithm repeats with l = m. If n < m2, then the square root must be less than m, so the algorithm repeats with r = m.

Either way, we have halved the interval with only one comparison. Therefore, after only lg2n rounds we will have identified the square root to within ±1.

Square Root (and other roots) of a Number

nr

Twenty QuestionsPlayer One - Picks a word.

Player Two - Asks a series of YES or NO questions, attempting to guess the word in <20 tries.

alphabetical version of Twenty Questions

Computing the Median

Computing the median of n numbers is easy: just sort them. The drawback is that this takes O(n log2n) time, whereas we would ideally like something linear. We have reason to be hopeful, because sorting is doing far more work than we really need. We just want the middle element and don't care about the relative ordering of the rest of them.

When looking for a recursive solution, it is paradoxically often easier to work with a more general version of the problem for the simple reason that this gives a more powerful step to recurse upon. In our case, the generalization we will consider is selection.

SELECTION

Input: A list of numbers S; an integer kOutput: The kth smallest element of S

For instance, if k = 1, the minimum of S is sought, whereas if k = [|S|/2], it is the median.

Divide-and-Conquer Algorithms - Vazirani

Chapter 2 Divide-and-conquer algorithms – see readingshttp://www.cs.berkeley.edu/~vazirani/

The K-Median Algorithm (Selection)

6 7 8 4 0 2 34 5 8 1 4 8 9 7 3

Finding the Median

6 7 8 402 3 4 5 81 4 8 9 73

SL SM SR

6 7 8 4 4 5 8 4 8 9 7

3

6 7 84 4 5 84 8 9 7

5

SL SM SR

4 4 44

Summary

Divide-and-Conquer (actually three steps)

1. divide 2. conquer 3. recombine

Most Naturally Implemented using Recursion (although iteration possible)

Analyze performance by solving Recurrence Relations

Worst-Case Performance may not be typical (or even possible) performance.

Divide-and-Conquer is embedded in Arithmetic Processors/Calculators

Cryptography relies on D&C Methods

Game Strategy can be based on D&C

Using D&C Selection Algorithm to Find the Median