Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication...

27
Spring 2014 Divide-and-Conquer 1 Divide-and-Conquer 7 2 9 4 2 4 7 9 7 2 2 7 9 4 4 9 7 7 2 2 9 9 4 4

Transcript of Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication...

Page 1: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 1

Divide-and-Conquer

7 2 ⏐ 9 4 → 2 4 7 9

7 ⏐ 2 → 2 7 9 ⏐ 4 → 4 9

7 → 7 2 → 2 9 → 9 4 → 4

Page 2: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 2

Outline and Reading ! Divide-and-conquer paradigm (§5.2) ! Review Merge-sort (§4.1.1) ! Recurrence Equations (§5.2.1)

n  Iterative substitution n  Recursion trees n  Guess-and-test n  The master method

! Integer Multiplication (§5.2.2)

Page 3: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 3

Divide-and-Conquer ! Divide-and conquer is a

general algorithm design paradigm: n  Divide: divide the input data S in

two or more disjoint subsets S1, S2, …

n  Recur: solve the subproblems recursively

n  Conquer: combine the solutions for S1, S2, …, into a solution for S

! The base case for the recursion are subproblems of constant size

! Analysis can be done using recurrence equations

Page 4: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 4

Merge-Sort Review ! Merge-sort on an input

sequence S with n elements consists of three steps: n  Divide: partition S into

two sequences S1 and S2 of about n/2 elements each

n  Recur: recursively sort S1 and S2

n  Conquer: merge S1 and S2 into a unique sorted sequence

Algorithm mergeSort(S, C) Input sequence S with n elements,

comparator C Output sequence S sorted

according to C if S.size() > 1

(S1, S2) ← partition(S, n/2) mergeSort(S1, C) mergeSort(S2, C) S ← merge(S1, S2)

Page 5: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 5

Recurrence Equation Analysis ! The conquer step of merge-sort consists of merging two sorted

sequences, each with n/2 elements and implemented by means of a doubly linked list, takes at most bn steps, for some constant b.

! Likewise, the basis case (n < 2) will take at b most steps. ! Therefore, if we let T(n) denote the running time of merge-sort:

! We can therefore analyze the running time of merge-sort by finding a closed form solution to the above equation. n  That is, a solution that has T(n) only on the left-hand side.

!"#

$+

<=

2if)2/(22if

)(nbnnTnb

nT

Page 6: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 6

Iterative Substitution ! In the iterative substitution, or “plug-and-chug,” technique, we

iteratively apply the recurrence equation to itself and see if we can find a pattern:

! Note that base, T(1)=b, case occurs when 2i=n. That is, i = log n. ! So,

! Thus, T(n) is O(n log n).

ibnnT

bnnTbnnTbnnT

bnnbnTbnnTnT

ii +=

=

+=

+=

+=

++=

+=

)2/(2...

4)2/(23)2/(22)2/(2

))2/())2/(2(2)2/(2)(

44

33

22

2

nbnbnnT log)( +=

Page 7: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 7

The Recursion Tree ! Draw the recursion tree for the recurrence relation and look for a

pattern:

depth T’s size

0 1 n

1 2 n/2

i 2i n/2i

… … …

!"#

$+

<=

2if)2/(22if

)(nbnnTnb

nT

time

bn

bn

bn

Total time = bn + bn log n (last level plus all previous levels)

Page 8: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 8

Guess-and-Test Method ! In the guess-and-test method, we guess a closed form solution

and then try to prove it is true by induction:

! Guess: T(n) < cn log n.

! Wrong: we cannot make this last line be less than cn log n

nbncnncnnbnncnnbnnnc

nbnnTnT

logloglog)2log(loglog))2/log()2/((2

log)2/(2)(

+!=

+!=

+=

+=

!"#

$+

<=

2iflog)2/(22if

)(nnbnnTnb

nT

Page 9: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 9

Guess-and-Test Method, Part 2 ! Recall the recurrence equation:

! Guess #2: T(n) < cn log2 n.

n  if c > b. ! So, T(n) is O(n log2 n). ! In general, to use this method, you need to have a good guess

and you need to be good at induction proofs.

ncnnbncnncnncn

nbnncnnbnnnc

nbnnTnT

2

2

2

2

log

loglog2log

log)2log(log

log))2/(log)2/((2

log)2/(2)(

!

++"=

+"=

+=

+=

!"#

$+

<=

2iflog)2/(22if

)(nnbnnTnb

nT

Page 10: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 10

Master Method ! Many divide-and-conquer recurrence equations have

the form:

! The Master Theorem:

!"#

$+

<=

dnnfbnaTdnc

nTif)()/(if

)(

.1 somefor )()/( provided )),((is)(then),(is)(if 3.

)log(is)(then),log(is)(if 2.

)(is)(then),(is)(if 1.

log

1loglog

loglog

<!

"#

""

"

+

+

$

%%

&

&

nfbnafnfnTnnf

nnnTnnnfnnTnOnf

a

kaka

aa

b

bb

bb

Page 11: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 11

Master Method, Example 1 ! The form:

! The Master Theorem:

! Example:

!"#

$+

<=

dnnfbnaTdnc

nTif)()/(if

)(

.1 somefor )()/( provided )),((is)(then),(is)(if 3.

)log(is)(then),log(is)(if 2.

)(is)(then),(is)(if 1.

log

1loglog

loglog

<!

"#

""

"

+

+

$

%%

&

&

nfbnafnfnTnnf

nnnTnnnfnnTnOnf

a

kaka

aa

b

bb

bb

nnTnT += )2/(4)(

Solution: logba=2, so case 1 says T(n) is Θ(n2).

Page 12: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 12

Master Method, Example 2 ! The form:

! The Master Theorem:

! Example:

!"#

$+

<=

dnnfbnaTdnc

nTif)()/(if

)(

.1 somefor )()/( provided )),((is)(then),(is)(if 3.

)log(is)(then),log(is)(if 2.

)(is)(then),(is)(if 1.

log

1loglog

loglog

<!

"#

""

"

+

+

$

%%

&

&

nfbnafnfnTnnf

nnnTnnnfnnTnOnf

a

kaka

aa

b

bb

bb

nnnTnT log)2/(2)( +=

Solution: logba=1, so case 2 says T(n) is Θ(n log2 n).

Page 13: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 13

Master Method, Example 3 ! The form:

! The Master Theorem:

! Example:

!"#

$+

<=

dnnfbnaTdnc

nTif)()/(if

)(

.1 somefor )()/( provided )),((is)(then),(is)(if 3.

)log(is)(then),log(is)(if 2.

)(is)(then),(is)(if 1.

log

1loglog

loglog

<!

"#

""

"

+

+

$

%%

&

&

nfbnafnfnTnnf

nnnTnnnfnnTnOnf

a

kaka

aa

b

bb

bb

nnnTnT log)3/()( +=

Solution: logba=0, so case 3 says T(n) is Θ(n log n).

Page 14: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 14

Master Method, Example 4 ! The form:

! The Master Theorem:

! Example:

!"#

$+

<=

dnnfbnaTdnc

nTif)()/(if

)(

.1 somefor )()/( provided )),((is)(then),(is)(if 3.

)log(is)(then),log(is)(if 2.

)(is)(then),(is)(if 1.

log

1loglog

loglog

<!

"#

""

"

+

+

$

%%

&

&

nfbnafnfnTnnf

nnnTnnnfnnTnOnf

a

kaka

aa

b

bb

bb

2)2/(8)( nnTnT +=Solution: logba=3, so case 1 says T(n) is Θ(n3).

Page 15: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 15

Master Method, Example 5 ! The form:

! The Master Theorem:

! Example:

!"#

$+

<=

dnnfbnaTdnc

nTif)()/(if

)(

.1 somefor )()/( provided )),((is)(then),(is)(if 3.

)log(is)(then),log(is)(if 2.

)(is)(then),(is)(if 1.

log

1loglog

loglog

<!

"#

""

"

+

+

$

%%

&

&

nfbnafnfnTnnf

nnnTnnnfnnTnOnf

a

kaka

aa

b

bb

bb

3)3/(9)( nnTnT +=Solution: logba=2, so case 3 says T(n) is Θ(n3).

Page 16: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 16

Master Method, Example 6 ! The form:

! The Master Theorem:

! Example:

!"#

$+

<=

dnnfbnaTdnc

nTif)()/(if

)(

.1 somefor )()/( provided )),((is)(then),(is)(if 3.

)log(is)(then),log(is)(if 2.

)(is)(then),(is)(if 1.

log

1loglog

loglog

<!

"#

""

"

+

+

$

%%

&

&

nfbnafnfnTnnf

nnnTnnnfnnTnOnf

a

kaka

aa

b

bb

bb

1)2/()( += nTnTSolution: logba=0, so case 2 says T(n) is Θ(log n).

(binary search)

Page 17: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 17

Master Method, Example 7 ! The form:

! The Master Theorem:

! Example:

!"#

$+

<=

dnnfbnaTdnc

nTif)()/(if

)(

.1 somefor )()/( provided )),((is)(then),(is)(if 3.

)log(is)(then),log(is)(if 2.

)(is)(then),(is)(if 1.

log

1loglog

loglog

<!

"#

""

"

+

+

$

%%

&

&

nfbnafnfnTnnf

nnnTnnnfnnTnOnf

a

kaka

aa

b

bb

bb

nnTnT log)2/(2)( +=Solution: logba=1, so case 1 says T(n) is Θ(n).

(heap construction)

Page 18: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 18

Iterative “Proof” of the Master Theorem ! Using iterative substitution, let us see if

we can find a pattern:

M

M

)/()/(

)()/()/()/(

)()/()/()())/())/((

)()/()(

1

0

2233

22

2

!"

=

+=

+++=

++=

++=

+=

k

i

iikk bnfabnTa

nfbnafbnfabnTanfbnafbnTanfbnfbnaTa

nfbnaTnT

Page 19: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 19

Iterative “Proof” of the Master Theorem

! We then distinguish the three cases as n  The first term is dominant n  Each part of the summation is equally dominant n  The summation is a geometric series

!

!"

=

"

=

+=

+=

1)(log

0

log

1)(log

0

log

)/()1(

)/()1(

n

i

iia

n

i

iin

bb

bb

bnfaTn

bnfaTa

M

Page 20: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 20

Integer Multiplication ! Algorithm: Multiply two n-bit integers I and J.

n  Divide step: Split I and J into high-order and low-order bits

n  We can then define I*J by multiplying the parts and adding:

n  So, T(n) = 4T(n/2) + n, which implies T(n) is Θ(n2). n  But that is no better than the algorithm we learned in grade

school.

ln

h

ln

h

JJJIII+=

+=2/

2/

2

2

lln

hln

lhn

hh

ln

hln

h

JIJIJIJIJJIIJI

+++=

++=2/2/

2/2/

222

)2(*)2(*

where did this come from?

Page 21: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 21

An Improved Integer Multiplication Algorithm ! Algorithm: Multiply two n-bit integers I and J.

n  Divide step: Split I and J into high-order and low-order bits

n  Observe that there is a different way to multiply parts:

n  So, T(n) = 3T(n/2) + n, which implies T(n) is O(nlog23), by

the Master Theorem. n  Thus, T(n) is O(n1.585).

ln

h

ln

h

JJJIII+=

+=2/

2/

2

2

lln

hllhn

hh

lln

llhhhlhhlllhn

hh

lln

llhhhllhn

hh

JIJIJIJIJIJIJIJIJIJIJIJI

JIJIJIJJIIJIJI

+++=

++++!!+=

+++!!+=

2/

2/

2/

2)(2

2])[(2

2]))([(2*

requires only 3 mults!

Page 22: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 22

Matrix Multiplication

! Given n x n matrices X and Y, wish to compute the product Z=XY.

! Formula for doing this is

! This runs in O(n3) time n  In fact, multiplying an n x m by an m x q takes nmq

operations

kj

n

kikij YXZ !

"

=

=1

0

Page 23: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 23

Matrix Multiplication

DHCFLDGCEKBHAFJBGAEI

HGFE

DCBA

LKJI

+=

+=

+=

+=

!"

#$%

&!"

#$%

&=!

"

#$%

&

Page 24: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 24

Matrix Multiplication

! Using the decomposition on previous slide, we can computer Z using 8 recursively computed (n/2)x(n/2) matrices plus four additions that can be done in O(n2) time

! Thus T(n) = 8T(n/2) + bn2

! Still gives T(n) is Θ(n3)

Page 25: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 25

Strassen’s Algorithm ! If we define the matrices S1 through S7

as follows

))(())(())((

)()()()(

7

6

5

4

3

2

1

FECASHGDBSHEDAS

EGDSEDCSHBASHFAS

+!=

+!=

++=

!=

+=

+=

!=

Page 26: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 26

Strassen’s Algorithm ! Then we get the following:

! So now we can compute Z=XY using only seven recursive multiplications

5371

43

21

2465

SSSSLSSKSSJ

SSSSI

+!!=

+=

+=

!++=

Page 27: Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication (§5.2.2) Spring 2014 Divide-and-Conquer 3 Divide-and-Conquer ! Divide-and conquer

Spring 2014 Divide-and-Conquer 27

Strassen’s Algorithm ! This gives the relation T(n)=7T(n/2)+bn2 for some

b>0. ! By the Master Theorem, we can thus multiply two

n x n matrices in Θ(nlog7) time, which is approximately Θ(n2.808) . n  May not seem like much, but if you’re multiplying two

100 x 100 matrices: w  n3 is 1,000,000 w  n2.808 is 413,048

! With added complexity, there are algorithms to multiply matrices in as little as Θ(n2.376) time n  Reduces figures above to 56,494