Divide-and-Conquergsyoung/CS3310/CS3310Notes/DivideandConquer.pdfTopic: Divide and Conquer 23 3....

CS 331 Dr. Young

Divide and Conquer0

1

YoungCS 331 D&A of Algo.

Topic: Divide and Conquer 1

Divide-and-Conquer

General idea:

Divide a problem into subprograms of the same kind; solve subprograms using the same approach, and combine partial solution (if necessary).



List List 1 List 2n elements n/2 elements n/2 elements

min, max min1, max1 min2, max2

min = MIN ( min1, min2 )max = MAX ( max1, max2)

CS 331 Dr. Young

Divide and Conquer0

2



The Divide-and-Conquer algorithm:procedure Rmaxmin (i, j, fmax, fmin); // i, j are index #, fmax,

begin // fmin are output parameters case:

i = j: fmax ← fmin ← A (i);i = j –1: if A (i) < A (j) then fmax ← A (j);

fmin ← A (i); else fmax ← A (i);

fmin ← A (j);else: mid ← ;

call Rmaxmin (i, mid, gmax, gmin);call Rmaxmin (mid+1, j, hmax, hmin);fmax ← MAX (gmax, hmax);fmin ← MIN (gmin, hmin);

endend;

2

ji



Example: find max and min in the array: 22, 13, -5, -8, 15, 60, 17, 31, 47 ( n = 9 )

Index: 1 2 3 4 5 6 7 8 9Array: 22 13 -5 -8 15 60 17 31 47

Rmaxmin(1, 9, 60, -8)

1, 5, 22, -8 6, 9, 60, 17

1, 3, 22, -5 4, 5, 15, -8 6,7, 60, 17 8, 9, 47, 31

1,2, 22, 13 3, 3,-5, -5

(1)

(2)

(3) (4)

(5)

(6)

(7) (8)

CS 331 Dr. Young

Divide and Conquer0

3



The straightforward algorithm:

max ← min ← A (1);for i ← 2 to n do

if A (i) > max, max ← A (i);if A (i) < min, min ← A (i);

Key comparisons: 2(n – 1)

1. Find the maximum and minimum

The problem: Given a list of unordered n elements, find max and min



Analysis: For algorithm containing recursive calls, we can use recurrence relation to find its complexity

T(n) - # of comparisons needed for RmaxminRecurrence relation:

otherwisen

TnT

nnT

nnT

2)2

(2)(

21)(

10)(

222)

2(222)2)

8(2(2

22)2

(22)2)4

(2(2

2)2

(2)(

233

322

22

2

nT

nT

nT

nT

nTnT

CS 331 Dr. Young

Divide and Conquer0

4



Assume n = 2k for some integer k

25.1

212

)22()2(2

)222()2

(2

1

1211

1

n

nn

T

nT

kk

kkk

k



Theorem:

Claim:

1)(

1)(

nbnc

naT

nbnT where are a, b, c constants

They don’t have to be the same constant. We make them the same here for simplicity. It will not affect the overall result.

canO

cannLogO

canO

nTaLogc

c

)(

)(

)(

)(

CS 331 Dr. Young

Divide and Conquer0

5



Proof: Assume n = ck

bnc

nTanT )()(

) ) ( (

) ) ( ) ( ) (( ) (

) ) ( ) ( ) ((

) ) ( ) ( ) ((

) ( ) (

) 1 ( ) (

) ) ( (

) ( ) ) ( ( ) (

0

021

021

021

0

0

2

2

1

1

2

2

33

232

22

2

ik

i

kkk

kkk

k

kkk

k

k

k

k

kk

c

a bn

c

a

c

a

c

a bn

c

a bn

c

a

c

a

c

a bn

c

n b a

c

a

c

a

c

a bn b a

c

a

c

a

c

a bn

c

n T a

c

a

c

a bn

c

n T a

bn c

n ab

c

n b

c

n T a a

bnc

n ab

c

n T a bn

c

n b

c

n T a a n T



If a < c, is a constant

T(n) = bn · a constant T(n) = O(n)

if a = c,

ik

i c

a)(

0

1)(0

kc

a ik

i

)(

)1(

)1(1)(0

nLognO

nLogbn

kbnbnnT

c

k

i

CS 331 Dr. Young

Divide and Conquer0

6



If a > c, 1)(

1)()(

1

0

c

ac

a

c

ak

ik

i

)(

)(

1)(

1)()()(

1

0

aLog

aLog

nLogkk

k

ik

i

c

c

c

nO

nb

ababc

abn

c

ac

a

bnc

abnnT



2. Matrix multiplication

The problem:

Multiply two matrices A and B, each of size nn

nnnnnn

CBA

CS 331 Dr. Young

Divide and Conquer0

7



The traditional way:

use three for-loop

kj

n

kikij BAC

1

)()( 3nOnT



The Divide-and-Conquer way:

2221

1211

2221

1211

2221

1211

BB

B

BB

AA

A

AA

CC

C

CC

2222122122

2122112121

2212121112

2112111111

BABAC

BABAC

BABAC

BABAC

transform the problem of multiplying A and B, each of size [n×n] into 8 subproblems, each of size

22

nn

CS 331 Dr. Young

Divide and Conquer0

8



)(

)2

(8)(

3

2

nO

ann

TnT

which an2 is for addition

so, it is no improvement compared with the traditional way



111

111

111

111

111

111Example:

use Divide-and-Conquer way to solve it as following:

0000

0333

0333

0333

0000

0111

0111

0111

0000

0111

0111

0111

00

03

00

01

00

01

01

01

00

11

00

33

00

11

00

01

11

11

00

11

03

03

00

01

01

01

01

01

11

11

33

33

00

11

01

01

11

11

11

11

22

21

12

11

C

C

C

C

CS 331 Dr. Young

Divide and Conquer0

9



Strassen’s matrix multiplication: Discover a way to compute the Cij’s using 7 multiplications

and 18 additions or subtractions

))((

)(

)(

)(

)(

))((

12111121

221211

112122

221211

112221

22112211

BBAAU

BAAT

BBAS

BBAR

BAAQ

BBAAP

UQRPC

SQC

TRC

VTSPC

BBAAV

22

21

12

11

22212212 ))((



•Algorithm :

procedure Strassen (n, A, B, C) // n is size, A,B the input matrices, C output matrix

beginif n = 2,

else(cont.)

;

;

;

;

2222122122

2122112121

2212121112

2112111111

babaC

babaC

babaC

babaC

CS 331 Dr. Young

Divide and Conquer0

10



elsePartition A into 4 submatrices: ;Partition B into 4 submatrices: ;call Strassen ( ;

call Strassen (

call Strassen ( ;

call Strassen ( ;

call Strassen ( ;

22211211 ,,, AAAA22211211 ,,, BBBB

),,,2 22112211 PBBAAn

);,,,2 112221 QBAAn

),,,2 221211 RBBAn

),,,2 112122 SBBAn

),,,2 221211 TBAAn



call Strassen ( ;

call Strassen ( ;

end;

),,,2 12111121 UBBAAn

;

;

;

;

22

21

12

11

UQRPC

SQC

TRC

VTSPC

),,,2 22212212 VBBAAn

CS 331 Dr. Young

Divide and Conquer0

11



Analysis:

2

2)2

(7)(2

nb

nann

TnT

22223

3

222

2

2

)4

7()

4

7()

2(7

)4

7()

2(7

)2

(7)(

ananann

T

anann

T

ann

TnT



Assume n = 2k for some integer k

)()(

)(

)(7)4

7(7

)4

7(7

14

7

1)4

7(

7

1)4

7()

2(7

81.27

777

4

722

2

1

21

221

1

nOnO

ncbcnnb

ncnbcnb

ncb

anb

ann

T

Lg

LgLgLg

LgLgnLgnLgn

kk

k

k

kk

k

CS 331 Dr. Young

Divide and Conquer0

12



3. Integer multiplication

The problem:

Multiply two large integers (n digits)

The traditional way:

Use two loops, it takes O(n2) operations



The Divide-and-Conquer way:

Suppose x and y are large integers, divide x into two parts (a and b), and divide y into c and d.

x:

y:

a b baxn

210

c d dcyn

210

)(1010

)10)(10(

2

22

bcadbdac

dcbayxn

n

nn

So, transform the problem of multiplying two n-digit integersinto four subproblems of multiplying two -digit integers.

2

n

CS 331 Dr. Young

Divide and Conquer0

13



Worst-Case is:

However, it is same as the traditional way.

Instead, we write:

Worst-Case is:

)()(

)2

(4)(

242 nOnO

bnn

TnT

Log

)()(

)2

(3)(

58.132 nOnO

bnn

TnT

Log

)))(((1010

)(1010

2

2

bdacdcbabdac

bcadbdacyxn

n

nn



The algorithm by Divide-and-Conquer: function multiplication (x,y)

beginn = MAX ( # of digits in x, # of digits in y);if (x = 0) or (y = 0), return 0;else if (n = 1), return x*y in the usual way;

elsem = ;a = x divide 10m;b = x rem 10m;c = y divide 10m;d = y rem 10m;p1= MULTIPLICATION (a, c);p2= MULTIPLICATION (b, d);p3 = MULTIPLICATION (a+b, c+d);return ;

end;

2

n

)(1010 21322

1 ppppp mm

CS 331 Dr. Young

Divide and Conquer0

14



4. Merge SortThe problem:

Given a list of n numbers, sort them in non-decreasing order

idea: Split each part into 2 equal parts and sort each part using Mergesort, then merge the two sorted sub-lists into one sorted list.

The algorithm:procedure MergeSort (low, high)

beginif then

call MergeSort (low, mid);call MergeSort (mid+1, high);call Merge (low, mid, high);

end;

highlow

;2

highlow

mid



procedure Merge (low, mid, high)begin

while (i mid and j high) if A(i) < A(j), then U(k) A(i);

i++;else U(k) A(j);

j++;k++;

if (i > mid)move A(j) through A(high) to U(k) through U(high);

elsemove A(i) through A(mid) to U(k) through U(high);

for p low to highA(p) U(p);

end;

;,1, lowkmidjlowi

CS 331 Dr. Young

Divide and Conquer0

15



Analysis:Worst-Case for MergeSort is:

Best-Case & Average-Case? Merge sort pays no attention to the original order of the list, it will keep divide the list into half until sub-lists of length 1, then start merging.

Therefore Best-Case & Average-Case are the same as the Worst-Case. O(nLogn)

)(

)2

(2)(

nLognO

bnn

TnT



5. Quick Sort

The problem:

Same as Merge Sort

Compared with Merge Sort:

• Quick Sort also use Divide-and-Conquer strategy

• Quick Sort eliminates merging operations

CS 331 Dr. Young

Divide and Conquer0

16



How it works?

Use partition

Eg: Sort array 65, 70, 75, 80, 85, 60, 55, 50, 4565 70 75 80 85 60 55 50 45

60 55 50 45 70 75 80 85

55 50 45

50 45

45

75 80 85

80 85

85

Pivot item

65

60 65 70

55 60 65 70 75

55 60 65 70 75 80



The algorithm:procedure QuickSort (p, q)

beginif p < q,then call Partition (p, q, pivotposition);

call QuickSort (p, pivotposition –1);call QuickSort (pivotposition+1, q);

end;

procedure Partition (low, high, pivotposition)begin

v A(low);j low;for i (low + 1) to high

if A(i) < v,j++;A(i) A(j);

pivotposition j;A(low) A(pivotposition);

end;

CS 331 Dr. Young

Divide and Conquer0

17



Analysis:

Worst-Case:Call Partition O(n) times, each time takes O(n) steps. So O(n2), and it is worse than Merge Sort in the Worst-Case.

Best-Case: Split the list evenly each time.

)(nLognO



Average-Case:Assume pivot is selected randomly, so the probability of pivot being the kth element is equal, k.

C(n) = # of key comparison for n items

• average it over all k, (CA(n) is average performance)

• multiply both side by n

kn

kpivotprob ,1

)(

)1())1()1()0((2)1(

)(

nCCCnn

nCn

AAA

A

)()1()1( knCkCn

))()1((1

)1()(1

knCkCn

nnC A

n

kAA

CS 331 Dr. Young

Divide and Conquer0

18



• replace n by n-1

• subtract (2) from (1)

)2())2()0((2)2)(1(

)1()1(

nCCnn

nCn

AA

A

)1()1()1(2)(

)1(2)1(2)1()1()(

nCnnnCn

nCnnCnnCn

AA

AAA



• divide n(n + 1) for both sides

Best-Case = Average-Case

)()(

)(

)1(21

21

2

)1(

120

))1(

1

43

2

32

1(2

2

)1(

0)1(:

)1(

)1(2

)1(

)2(2

)1()2(

)3(2

2

)3(

)1(

)1(2

)1(

)2(2

1

)2(

)1(

)1(2)1(

1

)(

12

2

nLognOnC

LognO

LognLogdkkk

kk

k

nn

nC

CNote

nn

n

nn

n

nn

n

n

nC

nn

n

nn

n

n

nC

nn

n

n

nC

n

nC

A

ee

nn

k

n

k

A

A

A

A

AA

CS 331 Dr. Young

Divide and Conquer0

19



6. Selection (Project #2)The problem:

Given a list of n elements find the kth smallest oneNote: special cases: when k = 1, min

when k = n, max

The solving strategy:

Algorithm #1

Use sorting strategy like MergeSort, sort the whole list and then return the kth element of the sorted list

Both Best-Case and Worst-Case take O(n Log n)



Algorithm #2idea: use “partition” (from Quick Sort) procedure repeatedly until we find the kth smallest element.

For each iteration:If pivot position i = k Done!If pivot position i < k to select (k-i)th element in the 2nd sub-list.If pivot position i > k to select kth element in the 1st sub-list.

Best-Case is O(n) (when pivot is the kth smallest one)Worst-Case will call partition O(n) times, each time takes O(n), so Worst-Case is O(n2).

pivot

i = pivot position

1st sub-list 2nd sub-list

CS 331 Dr. Young

Divide and Conquer0

20



procedure Algorththm#2 (A, n, k) // A is array, n is # of array, k is key

beginm 1;j n;loop

call Partition (m, j, pivotposition);case:

k = pivotposition:return A(k);

k < pivotposition:j pivotposition – 1;

else:m pivotposition + 1;k k - pivotposition;

end loop;end;



Algorithm #3idea: Same as Algorithm#2

except it always find a special element of the list called MM (Median of Medians),and use MM as the pivot.

In class KLA activity to explain MM.

We can show that both Best-Case and Worst-Case take O(n)

CS 331 Dr. Young

Divide and Conquer0

21



By choosing pivot more carefully, we can obtain a selection algorithm with Worst-Case complexity O(n)

How: Make sure pivot v is chosen s.t. at least some fraction of the elements will be smaller than v and at least some other fraction of elements will be greater than v.

mm (median of medians)Non-decreasingOrder(all columns) The middle row is in

non-decreasing order

Elements mm

Elements mm

Example: 35 (n) elements divide into 7 (n/r) subsets of size 5 (r)



By choosing pivot more carefully, we can obtain a selection algorithm with Worst-Case complexity O(n)

Use mm (median of medians) rule to choose pivot

procedure Select2 (A, n, k)begin

if n r, then sort A and return the kth element;

divide A into subset of size r each, ignore excess

elements, and let be the set of

medians of the subsets;

r

n

r

nmmmM ,,, 21

r

n

CS 331 Dr. Young

Divide and Conquer0

22



Select2 ;

use “Partition” to partition A using v as the pivot;// Assume v is at pivotposition

case:k = pivotposition: return (v);

k < pivotposition: let S be the set of elementsA (1,…, pivotposition –1), return Select2 (S, pivotposition –1, k);

else: let R be the set of element A (pivotposition+1,…, n), return Select2 (R, n- pivotposition, k-pivotposition);

end case;end;

v )2,,(

r

n

r

nM



Analysis:• How many Mi’s (or ) mm?

At least

• How many elements (or ) mm?

At least

• How many elements > mm (or < mm)? At most

Assume r = 5

2

r

n

22 r

nr

)22

(

r

nrn

)24(75.0

2.17.05

45.1

55.1)2

53(

nn

nn

n

nn

nn

CS 331 Dr. Young

Divide and Conquer0

23



• Therefore, procedure Select2 the Worst-Case complexity is:

where c is chosen sufficiently large so that T(n) cn for n < 24.

Proof:

Use induction to show T(n) 20 cn (n 24)

IB (induction base):

cnnTn

TnT )4

3()

5()(

cnccncn

cTTnTn

2024

24)244

3()

5

24()(,24



IH (induction hypothesis):

Suppose

IS (induction solution):

When n = m;

T(n) = O(n) complexity of Select2 of n elements

mncnnT 2420)(

cn

cmmcm

c

cmmTm

TmT

204

320

520

)4

3()

5()(

Divide-and-Conquergsyoung/CS3310/CS3310Notes/DivideandConquer.pdfTopic: Divide and Conquer 23 3....

Documents

Transcript of Divide-and-Conquergsyoung/CS3310/CS3310Notes/DivideandConquer.pdfTopic: Divide and Conquer 23 3....