recsolx

8/8/2019 recsolx

http://slidepdf.com/reader/full/recsolx 1/4

Analyzing Recursive Algorithms !

A recursive algorithm can often be described by a recurrence equation that describes the overall runtime on a problem of size n in terms of the runtime on smaller inputs.

For divide-and-conquer algorithms, we get recurrences like:

T(n) =! (1) if n " c aT(n/b) + D(n) + C(n) otherwise

•! a = number of subproblems we divide the problem into•! n/b = size of the subproblems (in terms of n )•! D(n) = time to divide the size n problem into subproblems•! C(n) = time to combine the subproblem solutions to get the

answer for the problem of size n

where

Analyzing Recursive Algorithms !

For recursive algorithms such as computing the factorial of n,we get an expression like the following:

T(n) = 1! if n = 0 T(n-1) + D(n) + C(n) otherwise

•! n-1 = size of the subproblems (in terms of n )•! D(n) = time to divide the size n problem into subproblems (O(1))•! C(n) = time to combine the subproblem solutions to get the

answer for the problem of size n (O(1))

where

Solving Recurrences (Ch. 4) !We will use the following 3 methods to solve recurrences1. ! Backward Substitution: involves seeing a pattern and converting it to a

summation.2. ! Make a guess and prove using induction.3. ! Apply the "Master Theorem": If the recurrence has the form

T(n) = aT(n/b) + f(n) then there is a formula that can (often) be applied,given in Section 4-3.

To make the solutions simpler, we will

• ! assume base cases are constant, i.e., T(n) = ! (1) for n small enough.

Solving recurrence for n! ! Algorithm F(n)

Input: a positive integer nOutput: n!

1. ! if n=02. ! then return 13. ! else 4. ! return F(n-1) * n

T(n) = T(n-1) + 1T(0) = 0

We can solve this recurrence (ie, find an expression of the running timethat is not given in terms of itself) using a method known as backward substitution.

T(n) = T(n-1) + 1 subst T(n-1) = T(n-2) + 1 !

= [T(n-2) + 1] + 1 = T(n-2) + 2 !

! ! subst T(n-2) = T(n-3) + 1 !

=[T(n-3) + 1] + 2 = T(n-3) + 3… !

… !

=T(n-i)+i = … = T(n-n)+n = 0 + n !

Therefore, this algorithm has linear running !

time. !

Solving Recurrences: Backward Substitution !

Example : T(n) = 2T(n/2) + n (look familiar?)

T(n) = 2T(n/2) + n= 2[2T(n/4) + n/2] + n expand T(n/2)= 4T(n/4) + n + n simplify (4 = 2 2)

= 4[2T(n/8) + n/4] + n + n expand T(n/4)= 8T(n/8) + n + n + n simplify…notice a pattern?= . . . . . .

= 2 lgnT(n/2 lgn) + . . . + n + n + n after lgn iterations

= c2 lgn + nlgn

= cn + nlgn 2 lgn = n lg2 = n

= ! (nlgn)


Example : T(n) = 4T(n/2) + n

T(n) = 4T(n/2) + n= 4[4T(n/4) + n/2] + n /* expand T(n/2) */ = 16T(n/4) + 2n + n /* simplify 16 = 4 2*/

= 16[4T(n/8) + n/4] + 2n + n /* expand T(n/4) */= 64T(n/8) + 4n + 2n + n /* simplify 64 = 4 3*/= . . . . . .

= 4 lgnT(n/2 lgn) + . . . + 4n + 2n + n /* after lgn iterations */

= c4 lgn + n /* convert to summation */

= cn lg4 + n (2 lgn - 1) /* 4 lgn = n lg4 = n 2 */= cn 2 + n(n - 1) /* 2 lgn = n lg2 = n */= ! (n 2)

2 k

k = 0

lg n "1

# = 2 0+ 21

+ ... + 2 lg n "1

8/8/2019 recsolx



Example : T(n) = T(n/2) + 1

T(n) = T(n/2) + 1= [T(n/4) + 1] + 1 /* expand T(n/2) */ = T(n/4) + 2 /* simplify */= [T(n/8) + 1] + 2 /* expand T(n/4) */= T(n/8) + 3 /* simplify */= . . . . . .

= T(n/2 lgn) + lgn /* 2lgn

= nlg2

= n */

= c + lgn

= ! (lgn) ?? Which algorithm ??

Binary Search (non-recursive) ! Algorithm Binary-Search(A[1…n], k)

Input: a sorted array A of n comparable items and search key k Output: Index of array’s element that is equal to k or -1 if k not found

1. ! l = 1; r =n2. ! while l <= r do3. ! m = floor of (l + r)/24. ! if k = A[m] return m5. ! else if k < A[m] r = m – 16. ! else l = m + 17.! return -1

What is the running time of this algorithm for an input of size n? Arethere best and worst cases input instances?

Binary Search (recursive) ! Algorithm Binary-Search-Rec(A[1…n], k, l, r)

Input: a sorted array A of n comparable items, search key k, leftmostand rightmost index positions in A

Output: Index of array’s element that is equal to k or -1 if k not found1. ! if (l > r) return -12. ! else3. ! m = floor of (l + r)/24.! if k = A[m] return m5. ! else if k < A[m] return Binary-Search-Rec(A, k, l, m-1)6. ! else return Binary-Search-Rec(A, k, m+1, r)

What is the running time of this algorithm for an input of size n?

Guessing an Upper Bound !Give an upper bound on the recurrence: T(n) = 2T( #n/2 $ ) + n. !Guess T(n) = O(nlgn) by showing that T(n) " cnlgn for some c > 0. !

Assume T( "n/2#) " c "n/2# lg("n/2#). #

T(n) ! " ! 2(c"n/2#lg("n/2 #)) + n !

! " ! cnlg(n/2) + n !

! =! cnlgn - cnlg2 + n !

! = ! cnlgn - cn + n !

! " ! cnlgn ! ! ! for c >= 1. !

Mathematical induction with a good guess !

Suppose T(n) = 1 if n = 2, and T(n) = T(n/2) + ! (1) if n = 2 k , " for k > 1. !Show T(n) = lgn by induction on the exponent k. !

Basis: When k = 1, n = 2. T(2) = lg2 = 1.!

IHOP: Assume T(2 k ) = lg2 k for some constant k >1. "

Inductive step: Show T(2 k+1 ) = lg (2k+1 ) = k+1. !

T(2 k+1) = ! T(2 k) + 1 /* given */ !

! = ! (lg2 k ) + 1 /* by inductive hypothesis */ !

! = ! k + 1 /* lg2 k = k */ !

Mathematical induction with a good guess !

Suppose T(n) = 1 if n = 1, and T(n) = T(n-1) + ! (n) for n > 1. !Show T(n) = O(n 2) by induction. !

Basis: When n = 1. T(1) = 1 2 = 1. !

IHOP: Assume T(k) = k 2 for all n < k. "

Inductive step: Show T(k) = k 2.!

T(k) ! = ! T(k-1) + k /* given */ !

! =! (k-1) 2 + k /* by inductive hypothesis */ !

! =! k2 - k + 1 !

! " ! k2 for k > 1 !

8/8/2019 recsolx


Solving Recurrences: Master Method (§4.3) !

The master method provides a 'cookbook' method for so lvingrecurrences of a certain form.

Master Theorem : Let a ! 1 and b > 1 be constants, let f(n) be afunction, and let T(n) be defined on nonnegative integers as:

T(n) = aT(n/b) + f(n) !

Recall, a is the number of subproblems and n/b is the size of each

subproblem.

Then, T(n) can be bounded asymptotically as follows:

1. ! T(n) = ! (n logba) if f(n) = O(n logba- $) for some constant $ > 02. ! T(n) = ! (n logbalgn) if f(n) = ! (n logba)3. ! T(n) = ! f(n) if f(n) = %(n logba+ $) for some constant $ > 0

Solving Recurrences: Master Method !

Intuition: Compare f(n) with ! (n log ba). !

case 1: f (n) is "polynomially smaller than" ! (n logba)!

case 2: f (n) is "asymptotically equal to" ! (n log ba)!

case 3: f (n) is "polynomially larger than" ! (n log ba)!

What is logba? The number of times we divide a by b to reach O(1). !

Master Method Restated !Master Theorem : If T(n) = aT(n/b) + O(n d) for some constantsa ! 1, b > 1, d ! 0, then

! (n logba) if d < log ba (a > b d)T(n) = ! (n dlgn) if d = log ba (a = b d)

! (n d) if d > log ba (a < b d)

Why? The proof uses a recursion tree argument.

Review of Logarithms !A logarithm is an inverse exponential function. Saying b x = y is !

equivalent to saying log by = x. !

• ! properties of logarithms: logb(xy) = log bx + log bylogb (x/y) = log bx - log bylogbxa = alog bxlogba= log xa/log xba = b log

ba (e.g., n = 2 lgn = n lg2 )

lgk n = (lgn) k

lglg(n) = lg(lgn)

Recursion Tree for Merge-Sort !

c! ! ! if n = 1 !

2T(n/2) + cn! ! otherwise !T(n) = !

Recurrence for worst-case running time for Merge-Sort !

cn! ! ! cn!

cn/2 ! cn/2 ! ! ! cn!

cn/4 ! ! cn!cn/4 !cn/4 !cn/4 !

c! c! c! c! c! c! c!c! cn!

cnlgn + cn !

lgn + 1 levels !

(h = lgn)!


Master Theorem : Let a ! 1 and b > 1 be constants, let f(n) be afunction, and let T(n) be defined on nonnegative integers as:

T(n) = aT(n/b) + f(n) !

Then, T(n) can be bounded asymptotically as follows:

1. ! T(n) = ! (n logba) if f(n) = O(n logba- $) for some constant $ > 02. ! T(n) = ! (n logbalgn) if f(n) = ! (n logba)3. ! T(n) = ! f(n) if f(n) = %(n logba+ $) for some constant $ > 0

and if a(f(n/b)) " c(f(n)) for some positive constantc < 1 and all su fficiently large n.

Case 3 requires us to also show a(f(n/b)) " c(f(n)), the “regularity” condition.

The regularity condition always holds whenever f(n) = n k and f(n)= %(n logba+ $) , so we don’t need to check it when f(n) is a polynomial.

8/8/2019 recsolx



These 3 cases do not cover all the possibilities for f(n).

There is a gap between cases 1 and 2 when f(n) is smaller than n logba,but not polynomially smaller.

There is a gap between cases 2 and 3 when f(n) is larger than n logba, butnot polynomially larger.

If the function falls into one of these 2 gaps, or if the regularity conditioncan’t be shown to ho ld, then the master method can’t be used to solvethe recurrence.


A more general version of Case 2 follows:

T(n) = ! (n logbalgk+1 n) if f(n) = ! (n logbalgk n) for k ! 0

This case covers the gap between cases 2 and 3 in which f(n) islarger than n logba by only a polylog factor. We’ll see an example of this type of recurrence in class.


Example: T(n) = 9T(n/3) + n !

•! a = 9, b = 3, f(n) = n, n logba = n log 39 = n 2!

•! compare f(n) = n with n 2 !

! n = O(n 2 -$) (so f(n) is polynomially smaller than n logba )!

•! case 1 applies: T(n) = ! (n2)!

•! a = 1, b = 2, f(n) = 1, n logba = n log21 = n 0 = 1 !

•! compare f(n) = 1 with 1 !

! 1 = ! (n0) (so f(n) is polynomially equal to n logba )!

•! case 2 applies: T(n) = ! (n0lgn) = ! (lgn) !

Example: T(n) = T(n/2) + 1 !


Example: T(n) = T(n/2) + n 2!

•! a = 1, b = 2, f(n) = n 2, n logba = n log21 = n 0 = 1 !

•! compare f(n) = n 2 with 1 !

! n2 = %(n0+$) (so f(n) is polynomially larger) !

•! Since f(n) is a polynomial in n, case 3 holds and T(n) = ! (n2)

Example: T(n) = 4T(n/2) + n 2!

•! a = 4, b = 2, f(n) = n 2, n logba = n log24 = n 2 !

•! compare f(n) = n 2 with n 2 !

! n2 = ! (n2) (so f(n) is polynomially equal) !

•! Case 2 holds and T(n) = ! (n2lgn)




•! a = 7, b = 3, f(n) = n 2, n logba = n log 37 = n 1+ $ !

•! compare f(n) = n 2 with n 1+ $ !

! n2 = %(n1+$) (so f(n) is polynomially larger) !

•! Since f(n) is a polynomial in n, case 3 holds and T(n) = ! (n2)

•! a = 7, b = 2, f(n) = n 2, n log ba = n log27 = n 2+ $!

•! compare f(n) = n 2 with n 2+ $ !

! n2 = O(n2+$) (so f(n) is polynomially smaller) !

•! Case 1 holds and T(n) = ! (n log27)

Mini-Homework 4!to be worked in-class 2/5/09 !

Use the master method to give tight bounds for the running time of T(n) in each of the following recurrences. Specify which case of the master theorem holds for each problem.

a) T(n) = 2T(n/2) + n 3

b) T(n) = T(2n/3) + 1

c) T(n) = 16T(n/4) + n 2

d) T(n) = 5T(n/2) + n 2

e) T(n) = 5T(n/2) + n 3

recsolx

Documents

Transcript of recsolx