Divide and Conquer

Design and Analysis of Algorithms - Chapter 4 1

Divide and ConquerDivide and Conquer

The most well known algorithm design strategy:The most well known algorithm design strategy:1.1. Divide instance of problem into two or more Divide instance of problem into two or more

smaller instancessmaller instances2.2. Solve smaller instances recursivelySolve smaller instances recursively3.3. Obtain solution to original (larger) instance Obtain solution to original (larger) instance

by combining these solutionsby combining these solutions


Divide-and-conquer techniqueDivide-and-conquer technique

subproblem 2 of size n/2

subproblem 1 of size n/2

a solution to subproblem 1

a solution tothe original problem

a solution to subproblem 2

a problem of size n


Divide and Conquer ExamplesDivide and Conquer Examples

Sorting: mergesort and quicksortSorting: mergesort and quicksort Tree traversalsTree traversals Binary searchBinary search Matrix multiplication - Strassen’s algorithmMatrix multiplication - Strassen’s algorithm Convex hull - QuickHull algorithmConvex hull - QuickHull algorithm


General Divide and Conquer General Divide and Conquer recurrence:recurrence:

TT((nn) = ) = aTaT((n/bn/b) + ) + f f ((nn)) where where f f ((nn)) ∈∈ ΘΘ((nnkk))

1.1. a < ba < bkk T T((nn) ) ∈∈ ΘΘ((nnkk)) 2.2. a = ba = bkk T T((nn) ) ∈∈ ΘΘ((nnk k lg lg n n )) 3.3. a > ba > bkk T T((nn) ) ∈∈ ΘΘ((nnlog log b b aa))

Note:Note: the same results hold with O instead of the same results hold with O instead of ΘΘ..


MergesortMergesortAlgorithm:Algorithm: Split array A[1..Split array A[1..nn] in two and make copies of each half in] in two and make copies of each half in arrays B[1.. arrays B[1.. nn/2 ] and C[1.. /2 ] and C[1.. nn/2 ]/2 ] Sort arrays B and CSort arrays B and C Merge sorted arrays B and C into array A as follows:Merge sorted arrays B and C into array A as follows:• Repeat until no elements remain in one of the arrays:Repeat until no elements remain in one of the arrays:

– compare the first elements in the remaining unprocessed portions of compare the first elements in the remaining unprocessed portions of the arraysthe arrays

– copy the smaller of the two into A, while incrementing the index copy the smaller of the two into A, while incrementing the index indicating the unprocessed portion of that array indicating the unprocessed portion of that array

• Once all elements in one of the arrays are processed, copy Once all elements in one of the arrays are processed, copy the remaining unprocessed elements from the other array the remaining unprocessed elements from the other array into A.into A.


Mergesort ExampleMergesort Example 7 2 1 6 47 2 1 6 4


Efficiency of mergesortEfficiency of mergesort C(n) = 2 C(n/2) + CC(n) = 2 C(n/2) + Cmergemerge(n) for n>1, C(1)=0 (n) for n>1, C(1)=0 All cases have same efficiency: All cases have same efficiency: ΘΘ((n n log log nn) ) Number of comparisons is close to theoretical Number of comparisons is close to theoretical

minimum for comparison-based sorting: minimum for comparison-based sorting: • log log n n ! ≈ ! ≈ nn lg lg n n - 1.44- 1.44 n n

Space requirement: Space requirement: ΘΘ((nn) () (NOTNOT in-place) in-place) Can be implemented without recursion Can be implemented without recursion

(bottom-up)(bottom-up)


QuicksortQuicksort Select a Select a pivotpivot (partitioning element) (partitioning element) Rearrange the list so that all the elements in the Rearrange the list so that all the elements in the

positions before the pivot are smaller than or equal positions before the pivot are smaller than or equal to the pivot and those after the pivot are larger than to the pivot and those after the pivot are larger than the pivot (See algorithm the pivot (See algorithm Partition Partition in section 4.2) in section 4.2)

Exchange the pivot with the last element in the first Exchange the pivot with the last element in the first (i.e., (i.e., ≤ sublist) – the pivot is now in its final position≤ sublist) – the pivot is now in its final position

Sort the two sublistsSort the two sublistsp

A[i]≤p A[i]>p


The partition algorithmThe partition algorithm


Quicksort ExampleQuicksort Example 15 22 13 27 12 10 20 2515 22 13 27 12 10 20 25


Efficiency of quicksortEfficiency of quicksort Best caseBest case: split in the middle : split in the middle — — ΘΘ( ( n n log log nn) ) Worst caseWorst case: sorted array! — : sorted array! — ΘΘ( ( nn22) ) Average caseAverage case: random arrays : random arrays —— ΘΘ( ( n n log log nn)) Improvements:Improvements:• better pivot selection: median of three partitioning better pivot selection: median of three partitioning

avoids worst case in sorted filesavoids worst case in sorted files• switch to insertion sort on small subfilesswitch to insertion sort on small subfiles• elimination of recursionelimination of recursionthese combine to 20-25% improvementthese combine to 20-25% improvement

Considered the method of choice for internal Considered the method of choice for internal sorting for large files (sorting for large files (nn ≥ 10000) ≥ 10000)


Multiplication of large integersMultiplication of large integers To multiply two integers of nTo multiply two integers of n11 and n and n22 digits by a pen- digits by a pen-

and-pencil algorithm, we perform nand-pencil algorithm, we perform n11nn22 multiplicationsmultiplications

However, remark the example of 23*14, where However, remark the example of 23*14, where 23=223=2..101011+3+3..101000 and 14=1 and 14=1..101011+4+4..101000

In general: c=a*b=cIn general: c=a*b=c22..1010nn+c+c11

..1010n/2n/2+c+c00..101000 where where

cc22=a=a11*b*b11, c, c00=a=a00*b*b00, c, c11=(a=(a11+a+a00)*(b)*(b11+b+b00)-(c)-(c22+c+c00))aa00,a,a11,b,b00,b,b11 be the left and right parts of a and b. be the left and right parts of a and b.

Recurrence relationRecurrence relation EfficiencyEfficiency


Closest Pair ProblemClosest Pair Problem ProblemProblem: find closest points among : find closest points among n n onesones in in kk--

dimensional spacedimensional space Assume points are sorted by Assume points are sorted by xx-coordinate values-coordinate values Divide the points in two subsets Divide the points in two subsets SS11 and and SS22 of of nn/2 /2

points by drawing a vertical line at points by drawing a vertical line at x=cx=c Select the closest pair among the closest pair of the Select the closest pair among the closest pair of the

left subset (distance dleft subset (distance d11), of the closest pair of the ), of the closest pair of the right subset (distance dright subset (distance d22), and the closest pair with ), and the closest pair with points from both sidespoints from both sides

We need to examine a stripe of width 2d, where We need to examine a stripe of width 2d, where d=min[dd=min[d11,d,d22]]


Closest Pair Problem (2)Closest Pair Problem (2) Geometric explanationGeometric explanation

Recurrence relation T(n) = 2T(n/2) + M(n)Recurrence relation T(n) = 2T(n/2) + M(n) Efficiency in O(n logn)Efficiency in O(n logn) OptimalityOptimality


QuickHull Algorithm QuickHull Algorithm

Inspired by Quicksort compute Convex Hull:Inspired by Quicksort compute Convex Hull: Assume points are sorted by Assume points are sorted by xx-coordinate values-coordinate values Identify extreme points Identify extreme points PP11 and and PP22 (part of hull) (part of hull) The set S of points is divided in two subsets SThe set S of points is divided in two subsets S11 and S and S22

Compute Convex Hull for SCompute Convex Hull for S11

Compute Convex Hull for SCompute Convex Hull for S22 in a similar manner in a similar manner

P1

P2

Pmax


QuickHull Algorithm (2) QuickHull Algorithm (2) How to compute the Convex (upper) Hull for SHow to compute the Convex (upper) Hull for S11

• find point find point PPmaxmax that is farthest away from line that is farthest away from line PP11PP22

• compute the hull of the points to the left of line compute the hull of the points to the left of line PP11PPmaxmax

• compute the hull of the points to the left of line compute the hull of the points to the left of line PPmaxmaxPP22

How to find the How to find the PPmaxmax point point• PPmaxmax maximizes the area of the triangle maximizes the area of the triangle PPmaxmaxPP11PP22

• if tie, select theif tie, select the P Pmaxmax that maximizes the angle that maximizes the angle PPmaxmaxPP11PP22

The points inside triangle The points inside triangle PPmaxmaxPP11PP2 2 can be excluded from can be excluded from further considerationfurther consideration

How to find geometrically the point How to find geometrically the point PPmaxmax and the points and the points to the left of line to the left of line PP11PPmaxmax


QuickHull Algorithm (3)QuickHull Algorithm (3) How to find geometrically the point How to find geometrically the point PPmaxmax and the points to and the points to

the left of line the left of line PP11PPmaxmax

• Given points PGiven points P11(x(x11,y,y11), P), P22(x(x22,y,y22), P), Pmaxmax(x(xmaxmax,y,ymaxmax))• The area of the triangle is half of the magnitude of the The area of the triangle is half of the magnitude of the

determinant determinant =x=x11yy22+x+xmaxmaxyy11+x+x22yymaxmax–x–xmaxmaxyy22–x–x22yy11-x-x11yymaxmax

• The sign of the expression is positive if and only if the The sign of the expression is positive if and only if the point Ppoint Pmaxmax is to the left of the line P is to the left of the line P11PP22

xx11 yy11 11

xx22 yy22 11

xxmaxmax yymaxmax 11


Efficiency of QuickHull Efficiency of QuickHull algorithmalgorithm Finding point farthest away from line Finding point farthest away from line PP11PP2 2 can be can be

done in linear timedone in linear time This gives same efficiency as quicksort: This gives same efficiency as quicksort: • Worst caseWorst case: : ΘΘ((nn22) ) • Average caseAverage case: : ΘΘ((n n log log nn))

If points are not initially sorted by x-coordinate If points are not initially sorted by x-coordinate value, this can be accomplished in value, this can be accomplished in ΘΘ((n n log log nn) — ) — no increase in asymptotic efficiency classno increase in asymptotic efficiency class

Other algorithms for convex hull:Other algorithms for convex hull:• Graham’s scanGraham’s scan• DCHullDCHullalso in also in ΘΘ((n n log log nn) )


Strassen’s matrix multiplicationStrassen’s matrix multiplication Strassen observed [1969] that the product of two matrices Strassen observed [1969] that the product of two matrices

A and B (of size 2A and B (of size 2nnx2x2nn) can be computed as follows:) can be computed as follows:

CC00 00 CC0101 A A0000 A A0101 B B0000 B B0101

= *= *

CC10 10 CC1111 A A1010 A A1111 B B1010 B B1111

MM11 + M + M44 - M - M5 5 + M+ M77 M M3 3 + M+ M55

= =

MM22 + M + M4 4 MM11 + M + M33 - M - M2 2 + M+ M66


Submatrices:Submatrices: MM11 = (A = (A0000 + A + A1111) * (B) * (B0000 + + BB1111)) MM22 = (A = (A1010 + A + A1111) * B) * B0000

MM33 = A = A0000 * (B * (B0101 - - BB1111)) MM44 = A = A1111 * (B * (B1010 - - BB0000)) MM55 = (A = (A0000 + A + A0101) * ) * BB1111 MM66 = (A = (A1010 - A - A0000) * (B) * (B0000 + + BB0101)) MM77 = (A = (A0101 - A - A1111) * (B) * (B1010 + + BB1111))


Efficiency of Strassen’s Efficiency of Strassen’s algorithmalgorithm If If n n is not a power of 2, matrices can be is not a power of 2, matrices can be

padded with zerospadded with zeros Number of multiplications:Number of multiplications: Number of additions:Number of additions: Recurrence relations:Recurrence relations: Other algorithms have improved this result, Other algorithms have improved this result,

but are even more complexbut are even more complex

Divide and Conquer

Documents

Transcript of Divide and Conquer