Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re...
Transcript of Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re...
![Page 1: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/1.jpg)
Divide and Conquer
Arash Rafiey
27 October, 2016
Arash Rafiey Divide and Conquer
![Page 2: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/2.jpg)
Divide the problem into a number of subproblems
Conquer the subproblems by solving them recursively or ifthey are small, there must be an easy solution.
Combine the solutions to the subproblems to the solution ofthe problem
Example:1. Merge-Sort(A, p, r)2. if p < r3. q := b(p + r)/2c4. Merge-Sort(A, p, q)5. Merge-Sort(A, q + 1, r)6. Merge(A, p, q, r)
Initial call for input array A = A[1] . . .A[n] isMerge-Sort(A, 1, n).
Arash Rafiey Divide and Conquer
![Page 3: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/3.jpg)
Divide the problem into a number of subproblems
Conquer the subproblems by solving them recursively or ifthey are small, there must be an easy solution.
Combine the solutions to the subproblems to the solution ofthe problem
Example:1. Merge-Sort(A, p, r)2. if p < r3. q := b(p + r)/2c4. Merge-Sort(A, p, q)5. Merge-Sort(A, q + 1, r)6. Merge(A, p, q, r)
Initial call for input array A = A[1] . . .A[n] isMerge-Sort(A, 1, n).
Arash Rafiey Divide and Conquer
![Page 4: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/4.jpg)
Divide the problem into a number of subproblems
Conquer the subproblems by solving them recursively or ifthey are small, there must be an easy solution.
Combine the solutions to the subproblems to the solution ofthe problem
Example:1. Merge-Sort(A, p, r)2. if p < r3. q := b(p + r)/2c4. Merge-Sort(A, p, q)5. Merge-Sort(A, q + 1, r)6. Merge(A, p, q, r)
Initial call for input array A = A[1] . . .A[n] isMerge-Sort(A, 1, n).
Arash Rafiey Divide and Conquer
![Page 5: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/5.jpg)
Divide the problem into a number of subproblems
Conquer the subproblems by solving them recursively or ifthey are small, there must be an easy solution.
Combine the solutions to the subproblems to the solution ofthe problem
Example:1. Merge-Sort(A, p, r)2. if p < r3. q := b(p + r)/2c4. Merge-Sort(A, p, q)5. Merge-Sort(A, q + 1, r)6. Merge(A, p, q, r)
Initial call for input array A = A[1] . . .A[n] isMerge-Sort(A, 1, n).
Arash Rafiey Divide and Conquer
![Page 6: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/6.jpg)
Divide the problem into a number of subproblems
Conquer the subproblems by solving them recursively or ifthey are small, there must be an easy solution.
Combine the solutions to the subproblems to the solution ofthe problem
Example:1. Merge-Sort(A, p, r)2. if p < r3. q := b(p + r)/2c4. Merge-Sort(A, p, q)5. Merge-Sort(A, q + 1, r)6. Merge(A, p, q, r)
Initial call for input array A = A[1] . . .A[n] isMerge-Sort(A, 1, n).
Arash Rafiey Divide and Conquer
![Page 7: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/7.jpg)
Example:
sorted sequence1 2 2 3 4 5 6 7
2 4 5 7 1 2 3 6
2 5 4 7 1 3 2 6
5 2 4 7 1 3 2 6initial sequence
Merge-Sort
splits length-` sequence into two length-`/2 sequencessorts them recursivelymerges the two sorted subsequences
Arash Rafiey Divide and Conquer
![Page 8: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/8.jpg)
Merge
Merge(A, p, q, r)
Take the smallest of the two front most elements of sequencesA[p..q] and A[q + 1..r ] and put it into a temporary array.
Repeat this, until both sequences are empty. Copy the resultingsequence from temporary array into A[p..r ].
Write a pseudo code for the procedure Merge(A, p, q, r) used inMerge-sort algorithm for merging two sorted arrays A[p . . . q] andA[q + 1 . . . r ] into one.
Arash Rafiey Divide and Conquer
![Page 9: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/9.jpg)
Merge
Merge(A, p, q, r)
Take the smallest of the two front most elements of sequencesA[p..q] and A[q + 1..r ] and put it into a temporary array.
Repeat this, until both sequences are empty. Copy the resultingsequence from temporary array into A[p..r ].
Write a pseudo code for the procedure Merge(A, p, q, r) used inMerge-sort algorithm for merging two sorted arrays A[p . . . q] andA[q + 1 . . . r ] into one.
Arash Rafiey Divide and Conquer
![Page 10: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/10.jpg)
MERGE(A, p, q, r)1. n1 := q − p + 1; n2 := r − q;2. for i := 1 to n1
3. L[i ] := A[p + i − 1]4. for i := 1 to n2
5. R[i ] := A[q + i ]6. i := 1; j := 1; k := p7. while i ≤ n1 and j ≤ n2
8. if L[i ] ≤ R[j ]9. A[k] := L[i ]; i := i + 1;10. else11. A[k] := R[j ]; j := j + 1;12. k := k + 1;13. if i > n1
14. while j ≤ n2
15. A[k] := R[j ]; j := j + 1; k := k + 1;16. while i ≤ n1
17. A[k] := L[i ]; i := i + 1; k := k + 1;
Arash Rafiey Divide and Conquer
![Page 11: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/11.jpg)
Analyzing an D&Q algorithm
Let T (n) be running time on problem of size n
If n is small enough (say, n ≤ c for constant c), thenstraightforward solution takes Θ(1)
If division of problem yields a subproblems, each of which 1/bof original (Merge-Sort: a = b = 2)
Division into subproblems takes D(n)
Combination of solutions to subproblems takes C (n)
Then,
T (n) =
Θ(1) if n ≤ caT (n/b) + D(n) + C (n) otherwise
Arash Rafiey Divide and Conquer
![Page 12: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/12.jpg)
Analyzing an D&Q algorithm
Let T (n) be running time on problem of size n
If n is small enough (say, n ≤ c for constant c), thenstraightforward solution takes Θ(1)
If division of problem yields a subproblems, each of which 1/bof original (Merge-Sort: a = b = 2)
Division into subproblems takes D(n)
Combination of solutions to subproblems takes C (n)
Then,
T (n) =
Θ(1) if n ≤ caT (n/b) + D(n) + C (n) otherwise
Arash Rafiey Divide and Conquer
![Page 13: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/13.jpg)
Analyzing an D&Q algorithm
Let T (n) be running time on problem of size n
If n is small enough (say, n ≤ c for constant c), thenstraightforward solution takes Θ(1)
If division of problem yields a subproblems, each of which 1/bof original (Merge-Sort: a = b = 2)
Division into subproblems takes D(n)
Combination of solutions to subproblems takes C (n)
Then,
T (n) =
Θ(1) if n ≤ caT (n/b) + D(n) + C (n) otherwise
Arash Rafiey Divide and Conquer
![Page 14: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/14.jpg)
Analyzing an D&Q algorithm
Let T (n) be running time on problem of size n
If n is small enough (say, n ≤ c for constant c), thenstraightforward solution takes Θ(1)
If division of problem yields a subproblems, each of which 1/bof original (Merge-Sort: a = b = 2)
Division into subproblems takes D(n)
Combination of solutions to subproblems takes C (n)
Then,
T (n) =
Θ(1) if n ≤ caT (n/b) + D(n) + C (n) otherwise
Arash Rafiey Divide and Conquer
![Page 15: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/15.jpg)
For Merge-Sort:
a = b = 2D(n) = Θ(1) (just compute “middle” of array)C (n) = Θ(n) (merging has running time linear in length ofresulting sequence)
Thus
TMS(n) =
Θ(1) if n ≤ 1TMS(bn/2c) + TMS(dn/2e) + Θ(n) otherwise.
That’s what’s called a recurrenceBut: we want closed form, i.e., we want to solve the recurrence.
Arash Rafiey Divide and Conquer
![Page 16: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/16.jpg)
Solving recurrences
There are a few methods for solving recurrences, some easy andnot powerful, some complicated and powerful.Methods:
1 guess & verify (also called“substitution method”)
2 master method3 generating functions
and some others.We’re going to see 1. and 2.
Arash Rafiey Divide and Conquer
![Page 17: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/17.jpg)
Substitution method
Basic idea:
1 “guess” the form of the solution
2 Use mathematical induction to find constants and show thatsolution works.
Usually more difficult part is the part 1.Back to example: we had
TMS(n) ≤ 2TMS(dn/2e) + Θ(n)
for n ≥ 2.If you hadn’t seen something like this before, how would youguess?There is no general way to guess the correct solutions. It takesexperience.
Arash Rafiey Divide and Conquer
![Page 18: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/18.jpg)
Heuristics that can help to find a good guess.
One way would be to have a look at first few terms. Say ifwe had T (n) = 2T (n/2) + 3n, then
T (n) = 2T (n/2) + 3n
= 2(2T (n/4) + 3(n/2)) + 3n
= 2(2(2T (n/8) + 3(n/4)) + 3(n/2)) + 3n
= 23T (n/23) + 223(n/22) + 213(n/21) + 203(n/20)
We can do this log n times
2log n · T (n/2log n) +
log(n)−1∑i=0
2i3(n/2i )
= n · T (1) + 3n ·log(n)−1∑
i=0
1
= n · T (1) + 3n log n = Θ(n log n)
After guessing a solution you’ll have to prove the correctness.
Arash Rafiey Divide and Conquer
![Page 19: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/19.jpg)
Heuristics that can help to find a good guess.
One way would be to have a look at first few terms. Say ifwe had T (n) = 2T (n/2) + 3n, then
T (n) = 2T (n/2) + 3n
= 2(2T (n/4) + 3(n/2)) + 3n
= 2(2(2T (n/8) + 3(n/4)) + 3(n/2)) + 3n
= 23T (n/23) + 223(n/22) + 213(n/21) + 203(n/20)
We can do this log n times
2log n · T (n/2log n) +
log(n)−1∑i=0
2i3(n/2i )
= n · T (1) + 3n ·log(n)−1∑
i=0
1
= n · T (1) + 3n log n = Θ(n log n)
After guessing a solution you’ll have to prove the correctness.Arash Rafiey Divide and Conquer
![Page 20: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/20.jpg)
similar recurrences might have similar solutionsConsider
T (n) = 2T (bn/2c+ 25) + n
Looks similar to last example, but is the additional 25 in theargument going to change the solution?Not really, because for large n, difference between
T (bn/2c) and T (bn/2c+ 25)
is not large: both cut n nearly in half: for n = 2, 000 we have
T (1, 000) and T (1, 025),
for n = 1, 000, 000 we have
T (500, 000) and T (500, 025).
Thus, reasonable assumption is that nowT (n) = O(n log n) as well.
Arash Rafiey Divide and Conquer
![Page 21: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/21.jpg)
stepwise refinement – guessing loose lower and upperbounds, and gradually taking them closer to each otherFor T (n) = 2T (bn/2c) + n we see
T (n) = Ω(n) (because of the n term)
T (n) = O(n2) (easily proven)
From there, we can perhaps “converge” on asymptoticallytight bound Θ(n log n).
Arash Rafiey Divide and Conquer
![Page 22: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/22.jpg)
stepwise refinement – guessing loose lower and upperbounds, and gradually taking them closer to each otherFor T (n) = 2T (bn/2c) + n we see
T (n) = Ω(n) (because of the n term)
T (n) = O(n2) (easily proven)
From there, we can perhaps “converge” on asymptoticallytight bound Θ(n log n).
Arash Rafiey Divide and Conquer
![Page 23: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/23.jpg)
stepwise refinement – guessing loose lower and upperbounds, and gradually taking them closer to each otherFor T (n) = 2T (bn/2c) + n we see
T (n) = Ω(n) (because of the n term)
T (n) = O(n2) (easily proven)
From there, we can perhaps “converge” on asymptoticallytight bound Θ(n log n).
Arash Rafiey Divide and Conquer
![Page 24: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/24.jpg)
Proving correctness of a guess
For Merge-Sort we have T (n) ≤ 2T (dn/2e) + Θ(n), which means,there is a constant d > 0 such that
T (n) ≤ 2T (dn/2e) + dn for n ≥ 2.
We guessed that T (n) = O(n log n).We prove T (n) ≤ cn log n for appropriate choice of constant c > 0.
Induction hypothesis: assume that bound holds for any n < m,hence also for n = dm/2e, i.e.,
T (dm/2e) ≤ cdm/2e log(dm/2e)
Arash Rafiey Divide and Conquer
![Page 25: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/25.jpg)
and prove it for m ≥ 4 (induction step):
T (m) ≤ 2T (dm/2e) + dm
≤ 2(cdm/2e log(dm/2e)) + dm
≤ 2(c(m + 1)/2 · log((m + 1)/2)) + dm
≤ c(m + 1) log((m + 1)/2) + dm
= c(m + 1) log(m + 1)− c(m + 1) log 2 + dm
≤ c(m + 1)(logm + 1/3)− c(m + 1) + dm
= cm logm + c logm + c/3(m + 1)− c(m + 1) + dm
≤ cm logm + cm/2− 2/3c(m + 1) + dm
≤ cm logm + (c/2− 2/3c + d)m = cm logm + (d − c/6)m
≤ cm logm for c ≥ 6d .
Arash Rafiey Divide and Conquer
![Page 26: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/26.jpg)
and prove it for m ≥ 4 (induction step):
T (m) ≤ 2T (dm/2e) + dm
≤ 2(cdm/2e log(dm/2e)) + dm
≤ 2(c(m + 1)/2 · log((m + 1)/2)) + dm
≤ c(m + 1) log((m + 1)/2) + dm
= c(m + 1) log(m + 1)− c(m + 1) log 2 + dm
≤ c(m + 1)(logm + 1/3)− c(m + 1) + dm
= cm logm + c logm + c/3(m + 1)− c(m + 1) + dm
≤ cm logm + cm/2− 2/3c(m + 1) + dm
≤ cm logm + (c/2− 2/3c + d)m = cm logm + (d − c/6)m
≤ cm logm for c ≥ 6d .
Arash Rafiey Divide and Conquer
![Page 27: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/27.jpg)
and prove it for m ≥ 4 (induction step):
T (m) ≤ 2T (dm/2e) + dm
≤ 2(cdm/2e log(dm/2e)) + dm
≤ 2(c(m + 1)/2 · log((m + 1)/2)) + dm
≤ c(m + 1) log((m + 1)/2) + dm
= c(m + 1) log(m + 1)− c(m + 1) log 2 + dm
≤ c(m + 1)(logm + 1/3)− c(m + 1) + dm
= cm logm + c logm + c/3(m + 1)− c(m + 1) + dm
≤ cm logm + cm/2− 2/3c(m + 1) + dm
≤ cm logm + (c/2− 2/3c + d)m = cm logm + (d − c/6)m
≤ cm logm for c ≥ 6d .
Arash Rafiey Divide and Conquer
![Page 28: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/28.jpg)
and prove it for m ≥ 4 (induction step):
T (m) ≤ 2T (dm/2e) + dm
≤ 2(cdm/2e log(dm/2e)) + dm
≤ 2(c(m + 1)/2 · log((m + 1)/2)) + dm
≤ c(m + 1) log((m + 1)/2) + dm
= c(m + 1) log(m + 1)− c(m + 1) log 2 + dm
≤ c(m + 1)(logm + 1/3)− c(m + 1) + dm
= cm logm + c logm + c/3(m + 1)− c(m + 1) + dm
≤ cm logm + cm/2− 2/3c(m + 1) + dm
≤ cm logm + (c/2− 2/3c + d)m = cm logm + (d − c/6)m
≤ cm logm for c ≥ 6d .
Arash Rafiey Divide and Conquer
![Page 29: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/29.jpg)
and prove it for m ≥ 4 (induction step):
T (m) ≤ 2T (dm/2e) + dm
≤ 2(cdm/2e log(dm/2e)) + dm
≤ 2(c(m + 1)/2 · log((m + 1)/2)) + dm
≤ c(m + 1) log((m + 1)/2) + dm
= c(m + 1) log(m + 1)− c(m + 1) log 2 + dm
≤ c(m + 1)(logm + 1/3)− c(m + 1) + dm
= cm logm + c logm + c/3(m + 1)− c(m + 1) + dm
≤ cm logm + cm/2− 2/3c(m + 1) + dm
≤ cm logm + (c/2− 2/3c + d)m = cm logm + (d − c/6)m
≤ cm logm for c ≥ 6d .
Arash Rafiey Divide and Conquer
![Page 30: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/30.jpg)
and prove it for m ≥ 4 (induction step):
T (m) ≤ 2T (dm/2e) + dm
≤ 2(cdm/2e log(dm/2e)) + dm
≤ 2(c(m + 1)/2 · log((m + 1)/2)) + dm
≤ c(m + 1) log((m + 1)/2) + dm
= c(m + 1) log(m + 1)− c(m + 1) log 2 + dm
≤ c(m + 1)(logm + 1/3)− c(m + 1) + dm
= cm logm + c logm + c/3(m + 1)− c(m + 1) + dm
≤ cm logm + cm/2− 2/3c(m + 1) + dm
≤ cm logm + (c/2− 2/3c + d)m = cm logm + (d − c/6)m
≤ cm logm for c ≥ 6d .
Arash Rafiey Divide and Conquer
![Page 31: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/31.jpg)
and prove it for m ≥ 4 (induction step):
T (m) ≤ 2T (dm/2e) + dm
≤ 2(cdm/2e log(dm/2e)) + dm
≤ 2(c(m + 1)/2 · log((m + 1)/2)) + dm
≤ c(m + 1) log((m + 1)/2) + dm
= c(m + 1) log(m + 1)− c(m + 1) log 2 + dm
≤ c(m + 1)(logm + 1/3)− c(m + 1) + dm
= cm logm + c logm + c/3(m + 1)− c(m + 1) + dm
≤ cm logm + cm/2− 2/3c(m + 1) + dm
≤ cm logm + (c/2− 2/3c + d)m = cm logm + (d − c/6)m
≤ cm logm for c ≥ 6d .
Arash Rafiey Divide and Conquer
![Page 32: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/32.jpg)
and prove it for m ≥ 4 (induction step):
T (m) ≤ 2T (dm/2e) + dm
≤ 2(cdm/2e log(dm/2e)) + dm
≤ 2(c(m + 1)/2 · log((m + 1)/2)) + dm
≤ c(m + 1) log((m + 1)/2) + dm
= c(m + 1) log(m + 1)− c(m + 1) log 2 + dm
≤ c(m + 1)(logm + 1/3)− c(m + 1) + dm
= cm logm + c logm + c/3(m + 1)− c(m + 1) + dm
≤ cm logm + cm/2− 2/3c(m + 1) + dm
≤ cm logm + (c/2− 2/3c + d)m = cm logm + (d − c/6)m
≤ cm logm for c ≥ 6d .
Arash Rafiey Divide and Conquer
![Page 33: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/33.jpg)
and prove it for m ≥ 4 (induction step):
T (m) ≤ 2T (dm/2e) + dm
≤ 2(cdm/2e log(dm/2e)) + dm
≤ 2(c(m + 1)/2 · log((m + 1)/2)) + dm
≤ c(m + 1) log((m + 1)/2) + dm
= c(m + 1) log(m + 1)− c(m + 1) log 2 + dm
≤ c(m + 1)(logm + 1/3)− c(m + 1) + dm
= cm logm + c logm + c/3(m + 1)− c(m + 1) + dm
≤ cm logm + cm/2− 2/3c(m + 1) + dm
≤ cm logm + (c/2− 2/3c + d)m = cm logm + (d − c/6)m
≤ cm logm for c ≥ 6d .
Arash Rafiey Divide and Conquer
![Page 34: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/34.jpg)
and prove it for m ≥ 4 (induction step):
T (m) ≤ 2T (dm/2e) + dm
≤ 2(cdm/2e log(dm/2e)) + dm
≤ 2(c(m + 1)/2 · log((m + 1)/2)) + dm
≤ c(m + 1) log((m + 1)/2) + dm
= c(m + 1) log(m + 1)− c(m + 1) log 2 + dm
≤ c(m + 1)(logm + 1/3)− c(m + 1) + dm
= cm logm + c logm + c/3(m + 1)− c(m + 1) + dm
≤ cm logm + cm/2− 2/3c(m + 1) + dm
≤ cm logm + (c/2− 2/3c + d)m = cm logm + (d − c/6)m
≤ cm logm for c ≥ 6d .
Arash Rafiey Divide and Conquer
![Page 35: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/35.jpg)
Base step
It remains to show that boundary conditions of recurrence (m < 4)are suitable as base cases for the inductive proof.We have got to show that we can choose c large enough s.t.bound
T (m) ≤ cm logm
works for boundary conditions as well (when m < 4).Assume that T (1) = b > 0.For m = 1,
T (m) ≤ cm logm = c · 1 · 0 = 0
is a bit of a problem, because T (1) is a constant greater than 0.
Arash Rafiey Divide and Conquer
![Page 36: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/36.jpg)
How to resolve this problem?Recall that we wanted to prove that T (n) = O(n log n).Also, recall that by def of O(), we are free to disregard a constantnumber of small values of n: f (n) = O(g(n))⇔there exist constants c , n0 : f (n) ≤ c · g(n) for n ≥ n0
A way out of our problem is to remove difficult boundarycondition T (1) = b > 0 from consideration in inductive proof.
Note: for m ≥ 4, T (m) does not depend directly on T (1)(T (2) ≤ 2T (1) + 2d = 2b + 2d ,T (3) ≤ 2T (2) + 3d = 2(2b + 2d) + 3d = 4b + 7d ,T (4) ≤ 2T (2) + 4d)
This means:We replace T (1) by T (2) and T (3) as the base case in theinductive proof, letting n0 = 2.
Arash Rafiey Divide and Conquer
![Page 37: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/37.jpg)
In other words, we are showing that the bound
T (n) ≤ cn log n
holds for any n ≥ 2.
For m = 2: T (2) = 2b + 2d? ≤?c .2 log 2 = 2c
For m = 3: T (3) = 4b + 7d? ≤?c .3 log 3
Hence, set c to max(6d , b + d , 4b+7d3 log 3 ) and the above boundary
conditions as well as the induction step will work.
Important:General technique! Can be used very often!
Arash Rafiey Divide and Conquer
![Page 38: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/38.jpg)
A neat trick called “changing variables”
Suppose we have
T (n) = 2T (√n) + log n
Now rename m = log n⇔ 2m = n. We know√n = n1/2 = (2m)1/2 = 2m/2 and thus obtain
T (2m) = 2T (2m/2) + m
Now rename S(m) = T (2m) and get
S(m) = 2S(m/2) + m
Looks familiar. We know the solution S(m) = Θ(m logm).Going back from S(m) to T (n) we obtain
T (n) = T (2m) = S(m) = Θ(m logm) = Θ(log n log log n)
Arash Rafiey Divide and Conquer
![Page 39: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/39.jpg)
A neat trick called “changing variables”
Suppose we have
T (n) = 2T (√n) + log n
Now rename m = log n⇔ 2m = n. We know√n = n1/2 = (2m)1/2 = 2m/2 and thus obtain
T (2m) = 2T (2m/2) + m
Now rename S(m) = T (2m) and get
S(m) = 2S(m/2) + m
Looks familiar. We know the solution S(m) = Θ(m logm).Going back from S(m) to T (n) we obtain
T (n) = T (2m) = S(m) = Θ(m logm) = Θ(log n log log n)
Arash Rafiey Divide and Conquer
![Page 40: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/40.jpg)
A neat trick called “changing variables”
Suppose we have
T (n) = 2T (√n) + log n
Now rename m = log n⇔ 2m = n. We know√n = n1/2 = (2m)1/2 = 2m/2 and thus obtain
T (2m) = 2T (2m/2) + m
Now rename S(m) = T (2m) and get
S(m) = 2S(m/2) + m
Looks familiar. We know the solution S(m) = Θ(m logm).
Going back from S(m) to T (n) we obtain
T (n) = T (2m) = S(m) = Θ(m logm) = Θ(log n log log n)
Arash Rafiey Divide and Conquer
![Page 41: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/41.jpg)
A neat trick called “changing variables”
Suppose we have
T (n) = 2T (√n) + log n
Now rename m = log n⇔ 2m = n. We know√n = n1/2 = (2m)1/2 = 2m/2 and thus obtain
T (2m) = 2T (2m/2) + m
Now rename S(m) = T (2m) and get
S(m) = 2S(m/2) + m
Looks familiar. We know the solution S(m) = Θ(m logm).Going back from S(m) to T (n) we obtain
T (n) = T (2m) = S(m) = Θ(m logm) = Θ(log n log log n)
Arash Rafiey Divide and Conquer
![Page 42: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/42.jpg)
Some basic algebraSums
a + (a + 1) + (a + 2) + · · ·+ (b − 1) + b =∑b
i=a i = (a+b)(b−a+1)2
a + a · c + a · c2 + · · ·+ a · cn−1 + a · cn =∑n
i=0 a · c i = a · 1−cn+1
1−c
if 0 < c < 1 then we can estimate the sum as follows∑ni=0 a · c i = a · 1−cn+1
1−c < a · 11−c
Arash Rafiey Divide and Conquer
![Page 43: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/43.jpg)
Some basic algebraSums
a + (a + 1) + (a + 2) + · · ·+ (b − 1) + b =∑b
i=a i = (a+b)(b−a+1)2
a + a · c + a · c2 + · · ·+ a · cn−1 + a · cn =∑n
i=0 a · c i = a · 1−cn+1
1−c
if 0 < c < 1 then we can estimate the sum as follows
∑ni=0 a · c i = a · 1−cn+1
1−c < a · 11−c
Arash Rafiey Divide and Conquer
![Page 44: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/44.jpg)
Some basic algebraSums
a + (a + 1) + (a + 2) + · · ·+ (b − 1) + b =∑b
i=a i = (a+b)(b−a+1)2
a + a · c + a · c2 + · · ·+ a · cn−1 + a · cn =∑n
i=0 a · c i = a · 1−cn+1
1−c
if 0 < c < 1 then we can estimate the sum as follows∑ni=0 a · c i = a · 1−cn+1
1−c < a · 11−c
Arash Rafiey Divide and Conquer
![Page 45: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/45.jpg)
logarithms and powers
loga n and an are inverse functions to each other:
loga(an) = n for all n
aloga n = n for all n > 0
properties:loga(b · c) = loga b + loga c
loga b = logc blogc a
loga b = 1logb a
alogc b = blogc a
Arash Rafiey Divide and Conquer
![Page 46: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/46.jpg)
logarithms and powers
loga n and an are inverse functions to each other:
loga(an) = n for all n
aloga n = n for all n > 0
properties:loga(b · c) = loga b + loga c
loga b = logc blogc a
loga b = 1logb a
alogc b = blogc a
Arash Rafiey Divide and Conquer
![Page 47: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/47.jpg)
logarithms and powers
loga n and an are inverse functions to each other:
loga(an) = n for all n
aloga n = n for all n > 0
properties:loga(b · c) = loga b + loga c
loga b = logc blogc a
loga b = 1logb a
alogc b = blogc a
Arash Rafiey Divide and Conquer
![Page 48: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/48.jpg)
logarithms and powers
loga n and an are inverse functions to each other:
loga(an) = n for all n
aloga n = n for all n > 0
properties:loga(b · c) = loga b + loga c
loga b = logc blogc a
loga b = 1logb a
alogc b = blogc a
Arash Rafiey Divide and Conquer
![Page 49: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/49.jpg)
logarithms and powers
loga n and an are inverse functions to each other:
loga(an) = n for all n
aloga n = n for all n > 0
properties:loga(b · c) = loga b + loga c
loga b = logc blogc a
loga b = 1logb a
alogc b = blogc a
Arash Rafiey Divide and Conquer
![Page 50: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/50.jpg)
logarithms and powers
loga n and an are inverse functions to each other:
loga(an) = n for all n
aloga n = n for all n > 0
properties:loga(b · c) = loga b + loga c
loga b = logc blogc a
loga b = 1logb a
alogc b = blogc a
Arash Rafiey Divide and Conquer
![Page 51: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/51.jpg)
logarithms and powers
loga n and an are inverse functions to each other:
loga(an) = n for all n
aloga n = n for all n > 0
properties:loga(b · c) = loga b + loga c
loga b = logc blogc a
loga b = 1logb a
alogc b = blogc a
Arash Rafiey Divide and Conquer
![Page 52: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/52.jpg)
The “Master Method”
Recipe for recurrences of the form
T (n) = aT (n/b) + f (n)
with a ≥ 1 and b > 1 constant, and f (n) an asymptoticallypositive function (f (n) = 5, f (n) = c log n, f (n) = n, f (n) = n12
are just fine).Split problem into a subproblems each of size n/b.Subproblems are solved recursively, each in time T (n/b).Dividing problem and combining solutions of subproblems iscaptured by f (n).Deals with many frequently seen recurrences (in particular, ourMerge-Sort example with a = b = 2 and f (n) = Θ(n)).
Arash Rafiey Divide and Conquer
![Page 53: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/53.jpg)
Theorem
Let a ≥ 1 and b > 1 be constants, let f (n) be a function, and letT (n) be defined on the nonnegative integers by the recurrence
T (n) = aT (n/b) + f (n),
where we interpret n/b to mean either bn/bc or dn/be. Then T (n)can be bounded asymptotically as follows.
1 If f (n) = O(n(logb a)−ε) for some constant ε > 0, thenT (n) = Θ(nlogb a).
2 If f (n) = Θ(nlogb a), thenT (n) = Θ(nlogb a · log n).
3 If f (n) = Ω(n(logb a)+ε) for some constant ε > 0, and ifa · f (n/b) ≤ c · f (n) for some constant c < 1 and allsufficiently large n, then T (n) = Θ(f (n)).
Arash Rafiey Divide and Conquer
![Page 54: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/54.jpg)
Notes on Master’s Theorem
2. If f (n) = Θ(nlogb a), thenT (n) = Θ(nlogb a · log n).
Note 1: Although it’s looking rather scary, it really isn’t. Forinstance, with Merge-Sort’s recurrence T (n) = 2T (n/2) + Θ(n)we have nlogb a = nlog2 2 = n1 = n, and we can apply case 2.The result is therefore Θ(nlogb a · log n) = Θ(n log n).
1 If f (n) = O(n(logb a)−ε) for some constant ε > 0, thenT (n) = Θ(nlogb a).
Note 2: In case 1,
f (n) = n(logb a)−ε = nlogb a/nε = o(nlogb a) ,
so the ε does matter. This case is basically about “small”functions f . But it’s not enough if f (n) is just asymptoticallysmaller than nlogb a (that is f (n) ∈ o(nlogb a), it must bepolynomially smaller!
Arash Rafiey Divide and Conquer
![Page 55: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/55.jpg)
Notes on Master’s Theorem
2. If f (n) = Θ(nlogb a), thenT (n) = Θ(nlogb a · log n).
Note 1: Although it’s looking rather scary, it really isn’t. Forinstance, with Merge-Sort’s recurrence T (n) = 2T (n/2) + Θ(n)we have nlogb a = nlog2 2 = n1 = n, and we can apply case 2.The result is therefore Θ(nlogb a · log n) = Θ(n log n).
1 If f (n) = O(n(logb a)−ε) for some constant ε > 0, thenT (n) = Θ(nlogb a).
Note 2: In case 1,
f (n) = n(logb a)−ε = nlogb a/nε = o(nlogb a) ,
so the ε does matter. This case is basically about “small”functions f . But it’s not enough if f (n) is just asymptoticallysmaller than nlogb a (that is f (n) ∈ o(nlogb a), it must bepolynomially smaller!
Arash Rafiey Divide and Conquer
![Page 56: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/56.jpg)
3. If f (n) = Ω(n(logb a)+ε) for some constant ε > 0, and ifa · f (n/b) ≤ c · f (n) for some constant c < 1 and allsufficiently large n, then T (n) = Θ(f (n)).
Note 3: Similarly, in case 3,
f (n) = n(logb a)+ε = nlogb a · nε = ω(nlogb a) ,
so the ε does matter again. This case is basically about “large”functions n. But again, f (n) ∈ ω(nlogb a) is not enough, it must bepolynomially larger. And in addition f (n) has to satisfy the“regularity condition”:
af (n/b) ≤ cf (n)
for some constant c < 1 and n ≥ n0 for some n0.
Arash Rafiey Divide and Conquer
![Page 57: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/57.jpg)
3. If f (n) = Ω(n(logb a)+ε) for some constant ε > 0, and ifa · f (n/b) ≤ c · f (n) for some constant c < 1 and allsufficiently large n, then T (n) = Θ(f (n)).
Note 3: Similarly, in case 3,
f (n) = n(logb a)+ε = nlogb a · nε = ω(nlogb a) ,
so the ε does matter again. This case is basically about “large”functions n. But again, f (n) ∈ ω(nlogb a) is not enough, it must bepolynomially larger. And in addition f (n) has to satisfy the“regularity condition”:
af (n/b) ≤ cf (n)
for some constant c < 1 and n ≥ n0 for some n0.
Arash Rafiey Divide and Conquer
![Page 58: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/58.jpg)
The idea is that we compare nlogb a to f (n).
Result is (intuitively) determined by larger of two.
In case 1, nlogb a is larger, so result is Θ(nlogb a).
In case 3, f (n) is larger, so result is Θ(f (n)).
In case 2, both have same order, we multiply it by a logarithmicfactor, and result is Θ(nlogb a · log n) = Θ(f (n) · log n).
Important: Does not cover all possible cases.
For instance, there is a gap between cases 1 and 2 whenever f (n)is smaller than nlogb a but not polynomially smaller.
Arash Rafiey Divide and Conquer
![Page 59: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/59.jpg)
The idea is that we compare nlogb a to f (n).
Result is (intuitively) determined by larger of two.
In case 1, nlogb a is larger, so result is Θ(nlogb a).
In case 3, f (n) is larger, so result is Θ(f (n)).
In case 2, both have same order, we multiply it by a logarithmicfactor, and result is Θ(nlogb a · log n) = Θ(f (n) · log n).
Important: Does not cover all possible cases.
For instance, there is a gap between cases 1 and 2 whenever f (n)is smaller than nlogb a but not polynomially smaller.
Arash Rafiey Divide and Conquer
![Page 60: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/60.jpg)
The idea is that we compare nlogb a to f (n).
Result is (intuitively) determined by larger of two.
In case 1, nlogb a is larger, so result is Θ(nlogb a).
In case 3, f (n) is larger, so result is Θ(f (n)).
In case 2, both have same order, we multiply it by a logarithmicfactor, and result is Θ(nlogb a · log n) = Θ(f (n) · log n).
Important: Does not cover all possible cases.
For instance, there is a gap between cases 1 and 2 whenever f (n)is smaller than nlogb a but not polynomially smaller.
Arash Rafiey Divide and Conquer
![Page 61: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/61.jpg)
The idea is that we compare nlogb a to f (n).
Result is (intuitively) determined by larger of two.
In case 1, nlogb a is larger, so result is Θ(nlogb a).
In case 3, f (n) is larger, so result is Θ(f (n)).
In case 2, both have same order, we multiply it by a logarithmicfactor, and result is Θ(nlogb a · log n) = Θ(f (n) · log n).
Important: Does not cover all possible cases.
For instance, there is a gap between cases 1 and 2 whenever f (n)is smaller than nlogb a but not polynomially smaller.
Arash Rafiey Divide and Conquer
![Page 62: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/62.jpg)
The idea is that we compare nlogb a to f (n).
Result is (intuitively) determined by larger of two.
In case 1, nlogb a is larger, so result is Θ(nlogb a).
In case 3, f (n) is larger, so result is Θ(f (n)).
In case 2, both have same order, we multiply it by a logarithmicfactor, and result is Θ(nlogb a · log n) = Θ(f (n) · log n).
Important: Does not cover all possible cases.
For instance, there is a gap between cases 1 and 2 whenever f (n)is smaller than nlogb a but not polynomially smaller.
Arash Rafiey Divide and Conquer
![Page 63: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/63.jpg)
Using the master theoremSimple enough. Some examples:
T (n) = 9T (n/3) + n
We have a = 9, b = 3, f (n) = n. Thus, nlogb a = nlog3 9 = n2.Clearly, f (n) = O(nlog3(9)−ε) for ε = 1, so case 1 givesT (n) = Θ(n2).
T (n) = T (2n/3) + 1
We have a = 1, b = 3/2, and f (n) = 1, sonlogb a = nlog2/3 1 = n0 = 1.
Apply case 2 (f (n) = Θ(nlogb a) = Θ(1), result is T (n) = Θ(log n).
Arash Rafiey Divide and Conquer
![Page 64: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/64.jpg)
Using the master theoremSimple enough. Some examples:
T (n) = 9T (n/3) + n
We have a = 9, b = 3, f (n) = n. Thus, nlogb a = nlog3 9 = n2.Clearly, f (n) = O(nlog3(9)−ε) for ε = 1, so case 1 givesT (n) = Θ(n2).
T (n) = T (2n/3) + 1
We have a = 1, b = 3/2, and f (n) = 1, sonlogb a = nlog2/3 1 = n0 = 1.
Apply case 2 (f (n) = Θ(nlogb a) = Θ(1), result is T (n) = Θ(log n).
Arash Rafiey Divide and Conquer
![Page 65: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/65.jpg)
T (n) = 3T (n/4) + n log n
We have a = 3, b = 4, and f (n) = n log n, sonlogb a = nlog4 3 = O(n0.793).
Clearly, f (n) = n log n = Ω(n) and
thus also f (n) = Ω(nlogb(a)+ε)
for ε ≈ 0.2. Also,a · f (n/b) = 3(n/4) log(n/4) ≤ (3/4)n log n = c · f (n) for anyc = 3/4 < 1.
Thus we can apply case 3 with result T (n) = Θ(n log n).
Arash Rafiey Divide and Conquer
![Page 66: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/66.jpg)
Exercises
Problem
Give asymptotic upper and lower bounds for T (n) in each of thefollowing recurrences. Assume that T (n) is constant for n ≤ 2.
(a) T (n) = 2T (n/2) + n3 (b) T (n) = T (n − 2) + n
(c) T (n) = T (√n) + log log n (d) T (n) = 16T (n/4) + n2
(e) T (n) = 2T (n − 1) + log n
Arash Rafiey Divide and Conquer
![Page 67: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/67.jpg)
Integer Multiplication
We are given two n-bits numbers x , y and we want to computex × y .
The usual multiplication takes O(n2) times.
Can we do better than O(n2) ?
We can write x = x12n2 + x0 and y = y12
n2 + y0.
x0, x1, y0, y1 are n2 -bits numbers.
Arash Rafiey Divide and Conquer
![Page 68: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/68.jpg)
Integer Multiplication
We are given two n-bits numbers x , y and we want to computex × y .
The usual multiplication takes O(n2) times.
Can we do better than O(n2) ?
We can write x = x12n2 + x0 and y = y12
n2 + y0.
x0, x1, y0, y1 are n2 -bits numbers.
Arash Rafiey Divide and Conquer
![Page 69: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/69.jpg)
Integer Multiplication
We are given two n-bits numbers x , y and we want to computex × y .
The usual multiplication takes O(n2) times.
Can we do better than O(n2) ?
We can write x = x12n2 + x0 and y = y12
n2 + y0.
x0, x1, y0, y1 are n2 -bits numbers.
Arash Rafiey Divide and Conquer
![Page 70: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/70.jpg)
Integer Multiplication
We are given two n-bits numbers x , y and we want to computex × y .
The usual multiplication takes O(n2) times.
Can we do better than O(n2) ?
We can write x = x12n2 + x0 and y = y12
n2 + y0.
x0, x1, y0, y1 are n2 -bits numbers.
Arash Rafiey Divide and Conquer
![Page 71: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/71.jpg)
For simplicity we write xy instead of x × y .
xy = (x12n2 + x0)(y12
n2 + y0) = x1y12n + (x1y0 + x0y1)2
n2 + x0y0.
Now if we compute each of the x1y1, x0y0, x1y0, x0y1 then we cancompute xy .
T (n) ≤ 4T (n/2) + cn.
O(n) is the time takes to add two n-bits numbers.
Using the Master Method for a = 4, b = 2 and f (n) = cn weobtain :
T (n) ≤ O(nlog2 4) = O(n2).
Not good!!
Arash Rafiey Divide and Conquer
![Page 72: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/72.jpg)
For simplicity we write xy instead of x × y .
xy = (x12n2 + x0)(y12
n2 + y0) = x1y12n + (x1y0 + x0y1)2
n2 + x0y0.
Now if we compute each of the x1y1, x0y0, x1y0, x0y1 then we cancompute xy .
T (n) ≤ 4T (n/2) + cn.
O(n) is the time takes to add two n-bits numbers.
Using the Master Method for a = 4, b = 2 and f (n) = cn weobtain :
T (n) ≤ O(nlog2 4) = O(n2).
Not good!!
Arash Rafiey Divide and Conquer
![Page 73: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/73.jpg)
For simplicity we write xy instead of x × y .
xy = (x12n2 + x0)(y12
n2 + y0) = x1y12n + (x1y0 + x0y1)2
n2 + x0y0.
Now if we compute each of the x1y1, x0y0, x1y0, x0y1 then we cancompute xy .
T (n) ≤ 4T (n/2) + cn.
O(n) is the time takes to add two n-bits numbers.
Using the Master Method for a = 4, b = 2 and f (n) = cn weobtain :
T (n) ≤ O(nlog2 4) = O(n2).
Not good!!
Arash Rafiey Divide and Conquer
![Page 74: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/74.jpg)
For simplicity we write xy instead of x × y .
xy = (x12n2 + x0)(y12
n2 + y0) = x1y12n + (x1y0 + x0y1)2
n2 + x0y0.
Now if we compute each of the x1y1, x0y0, x1y0, x0y1 then we cancompute xy .
T (n) ≤ 4T (n/2) + cn.
O(n) is the time takes to add two n-bits numbers.
Using the Master Method for a = 4, b = 2 and f (n) = cn weobtain :
T (n) ≤ O(nlog2 4) = O(n2).
Not good!!
Arash Rafiey Divide and Conquer
![Page 75: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/75.jpg)
If we compute p = (x1 + x0)(y1 + y0) = x1y1 + x1y0 + x0y1 + x0y0
then instead of x1y0 + x0y1 we can write p − x1y1 − x0y0.
p is the multiplication of two n2 -bits numbers.
So we need to compute the multiplication of three pairs of n2 -bits
numbers.
Arash Rafiey Divide and Conquer
![Page 76: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/76.jpg)
If we compute p = (x1 + x0)(y1 + y0) = x1y1 + x1y0 + x0y1 + x0y0
then instead of x1y0 + x0y1 we can write p − x1y1 − x0y0.
p is the multiplication of two n2 -bits numbers.
So we need to compute the multiplication of three pairs of n2 -bits
numbers.
Arash Rafiey Divide and Conquer
![Page 77: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/77.jpg)
Recursive-Multiply(x,y)
1. x := x12n2 + x0 and y := y12
n2 + y0.
2. Compute x1 + x0 and y1 + y0
3. p := Recursive-Multiply (x0 + x1,y0 + y1)
4. x1y1 := Recursive-Multiply (x1,y1)
5. x0y0 := Recursive-Multiply (x0,y0)
6. Return x1y12n + (p − x1y1 − x0y0)2n2 + x0y0
T (n) ≤ 3T (n/2) + cn.T (n) ≤ O(nlog2 3) = O(n1.59).
Arash Rafiey Divide and Conquer
![Page 78: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/78.jpg)
Recursive-Multiply(x,y)
1. x := x12n2 + x0 and y := y12
n2 + y0.
2. Compute x1 + x0 and y1 + y0
3. p := Recursive-Multiply (x0 + x1,y0 + y1)
4. x1y1 := Recursive-Multiply (x1,y1)
5. x0y0 := Recursive-Multiply (x0,y0)
6. Return x1y12n + (p − x1y1 − x0y0)2n2 + x0y0
T (n) ≤ 3T (n/2) + cn.T (n) ≤ O(nlog2 3) = O(n1.59).
Arash Rafiey Divide and Conquer
![Page 79: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/79.jpg)
Smallest distance between pairs of points
We are give a set p1, p2, . . . , pn of n points in plane (2D plane).Find a pair of points with the smallest distance.
It is clear that we can solve the problem in O(n2).
Can we do better than O(n2) ?
Sort the points by x-coordinate and produce list Px (O(n log n)).
Sort the points by y -coordinate and produce list Py (O(n log n)).
Set Q be the set of points in the first n2 positions in Px and let R
be the rest of the points in Px .
Construct the lists Qx and Qy and Rx and Ry .
1. Recursively compute closest pair of points for Q, say q0, q1 and
2. Recursively compute closest pair of points for R, say r0, r1.
Arash Rafiey Divide and Conquer
![Page 80: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/80.jpg)
Smallest distance between pairs of points
We are give a set p1, p2, . . . , pn of n points in plane (2D plane).Find a pair of points with the smallest distance.
It is clear that we can solve the problem in O(n2).
Can we do better than O(n2) ?
Sort the points by x-coordinate and produce list Px (O(n log n)).
Sort the points by y -coordinate and produce list Py (O(n log n)).
Set Q be the set of points in the first n2 positions in Px and let R
be the rest of the points in Px .
Construct the lists Qx and Qy and Rx and Ry .
1. Recursively compute closest pair of points for Q, say q0, q1 and
2. Recursively compute closest pair of points for R, say r0, r1.
Arash Rafiey Divide and Conquer
![Page 81: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/81.jpg)
Smallest distance between pairs of points
We are give a set p1, p2, . . . , pn of n points in plane (2D plane).Find a pair of points with the smallest distance.
It is clear that we can solve the problem in O(n2).
Can we do better than O(n2) ?
Sort the points by x-coordinate and produce list Px (O(n log n)).
Sort the points by y -coordinate and produce list Py (O(n log n)).
Set Q be the set of points in the first n2 positions in Px and let R
be the rest of the points in Px .
Construct the lists Qx and Qy and Rx and Ry .
1. Recursively compute closest pair of points for Q, say q0, q1 and
2. Recursively compute closest pair of points for R, say r0, r1.
Arash Rafiey Divide and Conquer
![Page 82: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/82.jpg)
Smallest distance between pairs of points
We are give a set p1, p2, . . . , pn of n points in plane (2D plane).Find a pair of points with the smallest distance.
It is clear that we can solve the problem in O(n2).
Can we do better than O(n2) ?
Sort the points by x-coordinate and produce list Px (O(n log n)).
Sort the points by y -coordinate and produce list Py (O(n log n)).
Set Q be the set of points in the first n2 positions in Px and let R
be the rest of the points in Px .
Construct the lists Qx and Qy and Rx and Ry .
1. Recursively compute closest pair of points for Q, say q0, q1 and
2. Recursively compute closest pair of points for R, say r0, r1.
Arash Rafiey Divide and Conquer
![Page 83: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/83.jpg)
Smallest distance between pairs of points
We are give a set p1, p2, . . . , pn of n points in plane (2D plane).Find a pair of points with the smallest distance.
It is clear that we can solve the problem in O(n2).
Can we do better than O(n2) ?
Sort the points by x-coordinate and produce list Px (O(n log n)).
Sort the points by y -coordinate and produce list Py (O(n log n)).
Set Q be the set of points in the first n2 positions in Px and let R
be the rest of the points in Px .
Construct the lists Qx and Qy and Rx and Ry .
1. Recursively compute closest pair of points for Q, say q0, q1 and
2. Recursively compute closest pair of points for R, say r0, r1.
Arash Rafiey Divide and Conquer
![Page 84: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/84.jpg)
Smallest distance between pairs of points
We are give a set p1, p2, . . . , pn of n points in plane (2D plane).Find a pair of points with the smallest distance.
It is clear that we can solve the problem in O(n2).
Can we do better than O(n2) ?
Sort the points by x-coordinate and produce list Px (O(n log n)).
Sort the points by y -coordinate and produce list Py (O(n log n)).
Set Q be the set of points in the first n2 positions in Px and let R
be the rest of the points in Px .
Construct the lists Qx and Qy and Rx and Ry .
1. Recursively compute closest pair of points for Q, say q0, q1 and
2. Recursively compute closest pair of points for R, say r0, r1.
Arash Rafiey Divide and Conquer
![Page 85: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/85.jpg)
Smallest distance between pairs of points
We are give a set p1, p2, . . . , pn of n points in plane (2D plane).Find a pair of points with the smallest distance.
It is clear that we can solve the problem in O(n2).
Can we do better than O(n2) ?
Sort the points by x-coordinate and produce list Px (O(n log n)).
Sort the points by y -coordinate and produce list Py (O(n log n)).
Set Q be the set of points in the first n2 positions in Px and let R
be the rest of the points in Px .
Construct the lists Qx and Qy and Rx and Ry .
1. Recursively compute closest pair of points for Q, say q0, q1 and
2. Recursively compute closest pair of points for R, say r0, r1.
Arash Rafiey Divide and Conquer
![Page 86: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/86.jpg)
Let δ be the smallest of d(q0, q1), d(r0, r1) (d(x , y) denote thedistance between x , y ).
Let x∗ denote the coordinates of the rightmost point in Q and letL be the line x∗ = x .L separates Q from R.
If there exist q ∈ Q and r ∈ R with d(q, r) < δ then each of q, rlies in distance at most δ of L.
So we can narrow down our search...Let S be the set of points in P in distance δ from L.
Let Sy denote the list of points in S sorted by increasingy -coordinate. (by going through list Py , we can compute Sy inO(n).)
Arash Rafiey Divide and Conquer
![Page 87: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/87.jpg)
Let δ be the smallest of d(q0, q1), d(r0, r1) (d(x , y) denote thedistance between x , y ).
Let x∗ denote the coordinates of the rightmost point in Q and letL be the line x∗ = x .L separates Q from R.
If there exist q ∈ Q and r ∈ R with d(q, r) < δ then each of q, rlies in distance at most δ of L.
So we can narrow down our search...Let S be the set of points in P in distance δ from L.
Let Sy denote the list of points in S sorted by increasingy -coordinate. (by going through list Py , we can compute Sy inO(n).)
Arash Rafiey Divide and Conquer
![Page 88: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/88.jpg)
Let δ be the smallest of d(q0, q1), d(r0, r1) (d(x , y) denote thedistance between x , y ).
Let x∗ denote the coordinates of the rightmost point in Q and letL be the line x∗ = x .L separates Q from R.
If there exist q ∈ Q and r ∈ R with d(q, r) < δ then each of q, rlies in distance at most δ of L.
So we can narrow down our search...Let S be the set of points in P in distance δ from L.
Let Sy denote the list of points in S sorted by increasingy -coordinate. (by going through list Py , we can compute Sy inO(n).)
Arash Rafiey Divide and Conquer
![Page 89: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/89.jpg)
Let δ be the smallest of d(q0, q1), d(r0, r1) (d(x , y) denote thedistance between x , y ).
Let x∗ denote the coordinates of the rightmost point in Q and letL be the line x∗ = x .L separates Q from R.
If there exist q ∈ Q and r ∈ R with d(q, r) < δ then each of q, rlies in distance at most δ of L.
So we can narrow down our search...Let S be the set of points in P in distance δ from L.
Let Sy denote the list of points in S sorted by increasingy -coordinate. (by going through list Py , we can compute Sy inO(n).)
Arash Rafiey Divide and Conquer
![Page 90: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/90.jpg)
There exist q ∈ Q and r ∈ R for which d(q, r) < δ if and only ifthere exist s, s ′ ∈ S for which d(s, s ′) < δ.
Now partition S into grids of size δ/2 by δ/2 in such a way that Lis one of the lines of the grid.
We note that in each of the grid cells there is at most one point.Otherwise the distance between two points in that grid cell is atmost δ
√2/2 < δ. A contradiction to δ = mind(q0, q1), d(r0, r1).
If s, s ′ ∈ S have the property that d(s, s ′) < δ then s, s ′ are within15 positions of each other in the sorted list Sy .
Arash Rafiey Divide and Conquer
![Page 91: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/91.jpg)
There exist q ∈ Q and r ∈ R for which d(q, r) < δ if and only ifthere exist s, s ′ ∈ S for which d(s, s ′) < δ.
Now partition S into grids of size δ/2 by δ/2 in such a way that Lis one of the lines of the grid.
We note that in each of the grid cells there is at most one point.Otherwise the distance between two points in that grid cell is atmost δ
√2/2 < δ. A contradiction to δ = mind(q0, q1), d(r0, r1).
If s, s ′ ∈ S have the property that d(s, s ′) < δ then s, s ′ are within15 positions of each other in the sorted list Sy .
Arash Rafiey Divide and Conquer
![Page 92: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/92.jpg)
ClosestPair(P)1. Construct Px and Py ( O(n log n) time)
2. (p∗0 , p∗1) = Closest-Pair(Px ,Py )
Closest-Pair(Px ,Py )1. If |P| ≤ 3 compute the closest pairs by trying all and return...
2. Construct Qx ,Qy and Rx ,Ry ( O(n) time )
3. (q∗0 , q∗1) = Closest − Pair(Qx ,Qy )
4. (r∗0 , r∗1 ) = Closest − Pair(Rx ,Ry )
5. δ := min(d(q∗0 , q∗1), d(r∗0 , r
∗1 )).
6.x∗ = maximum x-coordinate of a point in set Q
7. L = (x , y) : x = x∗
8. S = points in P within distance δ of L
9. Construct Sy
Arash Rafiey Divide and Conquer
![Page 93: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/93.jpg)
10. for each point s ∈ Sy , compute the distance from s to each ofnext 15 points in Sy .
11. Let s, s ′ be pair achieving minimum distance (O(n) time)
12. if d(s, s ′) < δ return (s, s ′)
13. if d(q∗0 , q∗1) < d(r∗0 , r
∗1 )) return (q∗0 , q
∗1)
14. else return (r∗0 , r∗1 )
Arash Rafiey Divide and Conquer
![Page 94: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/94.jpg)
Matrix Multiplication
Given : We are given two n × n matrixes A,B of integers.
Goal : We want to compute A× B.
Direct approach: has O(n3) time complexity.
Divide and Conquer:
Arash Rafiey Divide and Conquer
![Page 95: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/95.jpg)
C11 = A11 × B11 + A12 × B21
C12 = A11 × B12 + A12 × B22
C21 = A21 × B11 + A22 × B21
C22 = A21 × B12 + A22 × B22
In this case we have T (n) = 8T (n/2) +O(n2) = Θ(n3)How we can reduce the time complexity?
Arash Rafiey Divide and Conquer
![Page 96: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/96.jpg)
C11 = A11 × B11 + A12 × B21
C12 = A11 × B12 + A12 × B22
C21 = A21 × B11 + A22 × B21
C22 = A21 × B12 + A22 × B22
In this case we have T (n) = 8T (n/2) +O(n2) = Θ(n3)How we can reduce the time complexity?
Arash Rafiey Divide and Conquer
![Page 97: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/97.jpg)
Strassen’s Matrix Multiplication
In order to improve the algorithm we have to reduce the number ofmultiplication by introducing the new variables as follows:
P = (A11 + A22)× (B11 + B22)
Q = (A21 + A22)× B11
R = A11 × (B12 − B22)
S = A22 × (B21 − B11)
T = (A11 + A12)× B22
U = (A21 − A11)× (B11 + B12)
V = (A12 − A22)× (B21 + B22)
Now, we have:C11 = P + S − T + VC12 = R + TC21 = Q + SC22 = P + R − Q + U
So the T (n) can be expressed as:T (n) = 7T (n/2) + O(n2) = Θ(nlog2 7).
Arash Rafiey Divide and Conquer
![Page 98: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/98.jpg)
Strassen’s Matrix Multiplication
In order to improve the algorithm we have to reduce the number ofmultiplication by introducing the new variables as follows:
P = (A11 + A22)× (B11 + B22)
Q = (A21 + A22)× B11
R = A11 × (B12 − B22)
S = A22 × (B21 − B11)
T = (A11 + A12)× B22
U = (A21 − A11)× (B11 + B12)
V = (A12 − A22)× (B21 + B22)
Now, we have:C11 = P + S − T + VC12 = R + TC21 = Q + SC22 = P + R − Q + U
So the T (n) can be expressed as:T (n) = 7T (n/2) + O(n2) = Θ(nlog2 7).
Arash Rafiey Divide and Conquer
![Page 99: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/99.jpg)
Given an integer array of size n. Find the k-smallest number ?
Suppose n = 5(2r + 1) otherwise we add some zero to the end ofthe array.
Find the median for each group.
Arash Rafiey Divide and Conquer
![Page 100: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/100.jpg)
Given an integer array of size n. Find the k-smallest number ?
Suppose n = 5(2r + 1) otherwise we add some zero to the end ofthe array.
Find the median for each group.
Arash Rafiey Divide and Conquer
![Page 101: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/101.jpg)
Given an integer array of size n. Find the k-smallest number ?
Suppose n = 5(2r + 1) otherwise we add some zero to the end ofthe array.
Find the median for each group.
Arash Rafiey Divide and Conquer
![Page 102: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/102.jpg)
Arash Rafiey Divide and Conquer
![Page 103: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/103.jpg)
Arash Rafiey Divide and Conquer
![Page 104: Divide and Conquercs.indstate.edu/~arash/algolec10.pdfArash Ra ey Divide and Conquer stepwise re nement { guessing loose lower and upper bounds, and gradually taking them closer to](https://reader036.fdocuments.in/reader036/viewer/2022063019/5fe0dcd4c9b3130f002c52a7/html5/thumbnails/104.jpg)
Arash Rafiey Divide and Conquer