Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication...
Transcript of Divide-and-Conquerdszajda/classes/cs315/Spring_20… · The master method ! Integer Multiplication...
Spring 2014 Divide-and-Conquer 1
Divide-and-Conquer
7 2 ⏐ 9 4 → 2 4 7 9
7 ⏐ 2 → 2 7 9 ⏐ 4 → 4 9
7 → 7 2 → 2 9 → 9 4 → 4
Spring 2014 Divide-and-Conquer 2
Outline and Reading ! Divide-and-conquer paradigm (§5.2) ! Review Merge-sort (§4.1.1) ! Recurrence Equations (§5.2.1)
n Iterative substitution n Recursion trees n Guess-and-test n The master method
! Integer Multiplication (§5.2.2)
Spring 2014 Divide-and-Conquer 3
Divide-and-Conquer ! Divide-and conquer is a
general algorithm design paradigm: n Divide: divide the input data S in
two or more disjoint subsets S1, S2, …
n Recur: solve the subproblems recursively
n Conquer: combine the solutions for S1, S2, …, into a solution for S
! The base case for the recursion are subproblems of constant size
! Analysis can be done using recurrence equations
Spring 2014 Divide-and-Conquer 4
Merge-Sort Review ! Merge-sort on an input
sequence S with n elements consists of three steps: n Divide: partition S into
two sequences S1 and S2 of about n/2 elements each
n Recur: recursively sort S1 and S2
n Conquer: merge S1 and S2 into a unique sorted sequence
Algorithm mergeSort(S, C) Input sequence S with n elements,
comparator C Output sequence S sorted
according to C if S.size() > 1
(S1, S2) ← partition(S, n/2) mergeSort(S1, C) mergeSort(S2, C) S ← merge(S1, S2)
Spring 2014 Divide-and-Conquer 5
Recurrence Equation Analysis ! The conquer step of merge-sort consists of merging two sorted
sequences, each with n/2 elements and implemented by means of a doubly linked list, takes at most bn steps, for some constant b.
! Likewise, the basis case (n < 2) will take at b most steps. ! Therefore, if we let T(n) denote the running time of merge-sort:
! We can therefore analyze the running time of merge-sort by finding a closed form solution to the above equation. n That is, a solution that has T(n) only on the left-hand side.
!"#
$+
<=
2if)2/(22if
)(nbnnTnb
nT
Spring 2014 Divide-and-Conquer 6
Iterative Substitution ! In the iterative substitution, or “plug-and-chug,” technique, we
iteratively apply the recurrence equation to itself and see if we can find a pattern:
! Note that base, T(1)=b, case occurs when 2i=n. That is, i = log n. ! So,
! Thus, T(n) is O(n log n).
ibnnT
bnnTbnnTbnnT
bnnbnTbnnTnT
ii +=
=
+=
+=
+=
++=
+=
)2/(2...
4)2/(23)2/(22)2/(2
))2/())2/(2(2)2/(2)(
44
33
22
2
nbnbnnT log)( +=
Spring 2014 Divide-and-Conquer 7
The Recursion Tree ! Draw the recursion tree for the recurrence relation and look for a
pattern:
depth T’s size
0 1 n
1 2 n/2
i 2i n/2i
… … …
!"#
$+
<=
2if)2/(22if
)(nbnnTnb
nT
time
bn
bn
bn
…
Total time = bn + bn log n (last level plus all previous levels)
Spring 2014 Divide-and-Conquer 8
Guess-and-Test Method ! In the guess-and-test method, we guess a closed form solution
and then try to prove it is true by induction:
! Guess: T(n) < cn log n.
! Wrong: we cannot make this last line be less than cn log n
nbncnncnnbnncnnbnnnc
nbnnTnT
logloglog)2log(loglog))2/log()2/((2
log)2/(2)(
+!=
+!=
+=
+=
!"#
$+
<=
2iflog)2/(22if
)(nnbnnTnb
nT
Spring 2014 Divide-and-Conquer 9
Guess-and-Test Method, Part 2 ! Recall the recurrence equation:
! Guess #2: T(n) < cn log2 n.
n if c > b. ! So, T(n) is O(n log2 n). ! In general, to use this method, you need to have a good guess
and you need to be good at induction proofs.
ncnnbncnncnncn
nbnncnnbnnnc
nbnnTnT
2
2
2
2
log
loglog2log
log)2log(log
log))2/(log)2/((2
log)2/(2)(
!
++"=
+"=
+=
+=
!"#
$+
<=
2iflog)2/(22if
)(nnbnnTnb
nT
Spring 2014 Divide-and-Conquer 10
Master Method ! Many divide-and-conquer recurrence equations have
the form:
! The Master Theorem:
!"#
$+
<=
dnnfbnaTdnc
nTif)()/(if
)(
.1 somefor )()/( provided )),((is)(then),(is)(if 3.
)log(is)(then),log(is)(if 2.
)(is)(then),(is)(if 1.
log
1loglog
loglog
<!
"#
""
"
+
+
$
%%
&
&
nfbnafnfnTnnf
nnnTnnnfnnTnOnf
a
kaka
aa
b
bb
bb
Spring 2014 Divide-and-Conquer 11
Master Method, Example 1 ! The form:
! The Master Theorem:
! Example:
!"#
$+
<=
dnnfbnaTdnc
nTif)()/(if
)(
.1 somefor )()/( provided )),((is)(then),(is)(if 3.
)log(is)(then),log(is)(if 2.
)(is)(then),(is)(if 1.
log
1loglog
loglog
<!
"#
""
"
+
+
$
%%
&
&
nfbnafnfnTnnf
nnnTnnnfnnTnOnf
a
kaka
aa
b
bb
bb
nnTnT += )2/(4)(
Solution: logba=2, so case 1 says T(n) is Θ(n2).
Spring 2014 Divide-and-Conquer 12
Master Method, Example 2 ! The form:
! The Master Theorem:
! Example:
!"#
$+
<=
dnnfbnaTdnc
nTif)()/(if
)(
.1 somefor )()/( provided )),((is)(then),(is)(if 3.
)log(is)(then),log(is)(if 2.
)(is)(then),(is)(if 1.
log
1loglog
loglog
<!
"#
""
"
+
+
$
%%
&
&
nfbnafnfnTnnf
nnnTnnnfnnTnOnf
a
kaka
aa
b
bb
bb
nnnTnT log)2/(2)( +=
Solution: logba=1, so case 2 says T(n) is Θ(n log2 n).
Spring 2014 Divide-and-Conquer 13
Master Method, Example 3 ! The form:
! The Master Theorem:
! Example:
!"#
$+
<=
dnnfbnaTdnc
nTif)()/(if
)(
.1 somefor )()/( provided )),((is)(then),(is)(if 3.
)log(is)(then),log(is)(if 2.
)(is)(then),(is)(if 1.
log
1loglog
loglog
<!
"#
""
"
+
+
$
%%
&
&
nfbnafnfnTnnf
nnnTnnnfnnTnOnf
a
kaka
aa
b
bb
bb
nnnTnT log)3/()( +=
Solution: logba=0, so case 3 says T(n) is Θ(n log n).
Spring 2014 Divide-and-Conquer 14
Master Method, Example 4 ! The form:
! The Master Theorem:
! Example:
!"#
$+
<=
dnnfbnaTdnc
nTif)()/(if
)(
.1 somefor )()/( provided )),((is)(then),(is)(if 3.
)log(is)(then),log(is)(if 2.
)(is)(then),(is)(if 1.
log
1loglog
loglog
<!
"#
""
"
+
+
$
%%
&
&
nfbnafnfnTnnf
nnnTnnnfnnTnOnf
a
kaka
aa
b
bb
bb
2)2/(8)( nnTnT +=Solution: logba=3, so case 1 says T(n) is Θ(n3).
Spring 2014 Divide-and-Conquer 15
Master Method, Example 5 ! The form:
! The Master Theorem:
! Example:
!"#
$+
<=
dnnfbnaTdnc
nTif)()/(if
)(
.1 somefor )()/( provided )),((is)(then),(is)(if 3.
)log(is)(then),log(is)(if 2.
)(is)(then),(is)(if 1.
log
1loglog
loglog
<!
"#
""
"
+
+
$
%%
&
&
nfbnafnfnTnnf
nnnTnnnfnnTnOnf
a
kaka
aa
b
bb
bb
3)3/(9)( nnTnT +=Solution: logba=2, so case 3 says T(n) is Θ(n3).
Spring 2014 Divide-and-Conquer 16
Master Method, Example 6 ! The form:
! The Master Theorem:
! Example:
!"#
$+
<=
dnnfbnaTdnc
nTif)()/(if
)(
.1 somefor )()/( provided )),((is)(then),(is)(if 3.
)log(is)(then),log(is)(if 2.
)(is)(then),(is)(if 1.
log
1loglog
loglog
<!
"#
""
"
+
+
$
%%
&
&
nfbnafnfnTnnf
nnnTnnnfnnTnOnf
a
kaka
aa
b
bb
bb
1)2/()( += nTnTSolution: logba=0, so case 2 says T(n) is Θ(log n).
(binary search)
Spring 2014 Divide-and-Conquer 17
Master Method, Example 7 ! The form:
! The Master Theorem:
! Example:
!"#
$+
<=
dnnfbnaTdnc
nTif)()/(if
)(
.1 somefor )()/( provided )),((is)(then),(is)(if 3.
)log(is)(then),log(is)(if 2.
)(is)(then),(is)(if 1.
log
1loglog
loglog
<!
"#
""
"
+
+
$
%%
&
&
nfbnafnfnTnnf
nnnTnnnfnnTnOnf
a
kaka
aa
b
bb
bb
nnTnT log)2/(2)( +=Solution: logba=1, so case 1 says T(n) is Θ(n).
(heap construction)
Spring 2014 Divide-and-Conquer 18
Iterative “Proof” of the Master Theorem ! Using iterative substitution, let us see if
we can find a pattern:
M
M
)/()/(
)()/()/()/(
)()/()/()())/())/((
)()/()(
1
0
2233
22
2
!"
=
+=
+++=
++=
++=
+=
k
i
iikk bnfabnTa
nfbnafbnfabnTanfbnafbnTanfbnfbnaTa
nfbnaTnT
Spring 2014 Divide-and-Conquer 19
Iterative “Proof” of the Master Theorem
! We then distinguish the three cases as n The first term is dominant n Each part of the summation is equally dominant n The summation is a geometric series
!
!"
=
"
=
+=
+=
1)(log
0
log
1)(log
0
log
)/()1(
)/()1(
n
i
iia
n
i
iin
bb
bb
bnfaTn
bnfaTa
M
Spring 2014 Divide-and-Conquer 20
Integer Multiplication ! Algorithm: Multiply two n-bit integers I and J.
n Divide step: Split I and J into high-order and low-order bits
n We can then define I*J by multiplying the parts and adding:
n So, T(n) = 4T(n/2) + n, which implies T(n) is Θ(n2). n But that is no better than the algorithm we learned in grade
school.
ln
h
ln
h
JJJIII+=
+=2/
2/
2
2
lln
hln
lhn
hh
ln
hln
h
JIJIJIJIJJIIJI
+++=
++=2/2/
2/2/
222
)2(*)2(*
where did this come from?
Spring 2014 Divide-and-Conquer 21
An Improved Integer Multiplication Algorithm ! Algorithm: Multiply two n-bit integers I and J.
n Divide step: Split I and J into high-order and low-order bits
n Observe that there is a different way to multiply parts:
n So, T(n) = 3T(n/2) + n, which implies T(n) is O(nlog23), by
the Master Theorem. n Thus, T(n) is O(n1.585).
ln
h
ln
h
JJJIII+=
+=2/
2/
2
2
lln
hllhn
hh
lln
llhhhlhhlllhn
hh
lln
llhhhllhn
hh
JIJIJIJIJIJIJIJIJIJIJIJI
JIJIJIJJIIJIJI
+++=
++++!!+=
+++!!+=
2/
2/
2/
2)(2
2])[(2
2]))([(2*
requires only 3 mults!
Spring 2014 Divide-and-Conquer 22
Matrix Multiplication
! Given n x n matrices X and Y, wish to compute the product Z=XY.
! Formula for doing this is
! This runs in O(n3) time n In fact, multiplying an n x m by an m x q takes nmq
operations
kj
n
kikij YXZ !
"
=
=1
0
Spring 2014 Divide-and-Conquer 23
Matrix Multiplication
DHCFLDGCEKBHAFJBGAEI
HGFE
DCBA
LKJI
+=
+=
+=
+=
!"
#$%
&!"
#$%
&=!
"
#$%
&
Spring 2014 Divide-and-Conquer 24
Matrix Multiplication
! Using the decomposition on previous slide, we can computer Z using 8 recursively computed (n/2)x(n/2) matrices plus four additions that can be done in O(n2) time
! Thus T(n) = 8T(n/2) + bn2
! Still gives T(n) is Θ(n3)
Spring 2014 Divide-and-Conquer 25
Strassen’s Algorithm ! If we define the matrices S1 through S7
as follows
))(())(())((
)()()()(
7
6
5
4
3
2
1
FECASHGDBSHEDAS
EGDSEDCSHBASHFAS
+!=
+!=
++=
!=
+=
+=
!=
Spring 2014 Divide-and-Conquer 26
Strassen’s Algorithm ! Then we get the following:
! So now we can compute Z=XY using only seven recursive multiplications
5371
43
21
2465
SSSSLSSKSSJ
SSSSI
+!!=
+=
+=
!++=
Spring 2014 Divide-and-Conquer 27
Strassen’s Algorithm ! This gives the relation T(n)=7T(n/2)+bn2 for some
b>0. ! By the Master Theorem, we can thus multiply two
n x n matrices in Θ(nlog7) time, which is approximately Θ(n2.808) . n May not seem like much, but if you’re multiplying two
100 x 100 matrices: w n3 is 1,000,000 w n2.808 is 413,048
! With added complexity, there are algorithms to multiply matrices in as little as Θ(n2.376) time n Reduces figures above to 56,494