Transcript of Analysis of Recursive Algorithms October 29, 2014 1.
- Slide 1
- Slide 2
- Analysis of Recursive Algorithms October 29, 2014 1
- Slide 3
- Recursive Algorithms Analysis becomes a bit more complicated
for recursive algorithms. Recurrence relations is helpful for such
analyses. A recursive step can be expressed as a recurrence
relation, showing how earlier terms in a sequence are related to
later terms. 2
- Slide 4
- Recurrence Relation A recurrence relation for a sequence a 1, a
2, a 3, is a formula that relates each term a k to its certain
predecessors a k-1, a k-2, , a k-i where i is a fixed integer and k
is any integer greater than i. The initial conditions for such a
recurrence relation specify the values of a 1, a 2, , a i-1. 3
- Slide 5
- Example For the Tower of Hanoi problem: F 1 = 1 F n = 2F n-1 +
1 (It can be shown with a complex proof that 2 n -1 is the optimal
number of steps.) Fibonacci Sequence: F 0 = 0 F 1 = 1 F n = F n-1 +
F n-2 4
- Slide 6
- Class Discussion Can you set up some recurrence relations to
compute f(n) where f(n) is the number of 0-1 bit strings with n
bits and with no consecutive 1s, i.e., no 11 patterns? Can you set
up a recurrence relation to count the number of ways to
parenthesize x 1 x 2 x n ? E.g., when n = 3, there are two ways,
((x 1 x 2 ) x 3 ) and (x 1 (x 2 x 3 )). 5 f(n) = f 0 (n)+f 1 (n); f
0 (n) = f(n-1); f 1 (n) = f 0 (n-1); f 0 (1) = f 1 (1) = 1
- Slide 7
- Repeated Substitution Solving recurrence relation is the key to
analyze recursive algorithms. One direct method of solving
recurrence relations is called repeated substitution. This is done
by repeatedly substitute the formulae for the earlier terms in the
recurrence relation. 6
- Slide 8
- Repeated Substitution For example: a 0 = 1 a k = a k-1 + 2 We
start by: a k = a k-1 + 2 = a k-2 + 2 + 2 = a k-3 + 2 + 2 + 2 = a 0
+ 2k = 1 + 2k 7
- Slide 9
- Induction We can also start with a guess of the closed form
solution. Then, prove its correctness with induction. 8
- Slide 10
- Induction For example: a 0 = 1 a k = a k-1 + 2 Suppose we want
to show that a k =1+2k: Base case: k = 0, a 0 = 1 = 1 + 2 0
Inductive case: Assume that a k = 1+2k for k 0 a k+1 = a k + 2 = 1
+ 2k + 2 = 1 + 2(k+1) Therefore, we can conclude that a k =1+2k
(Structural Induction) 9
- Slide 11
- Class Discussion Give a closed form solution to the following
recurrence relation: a 0 = 5 a n = na n-1 10
- Slide 12
- Recursive Algorithms Solving recurrence relations is the key to
analyze recursive algorithms. A recurrence relation can be used to
express the running time of a recursive algorithm, showing how many
recursive calls are made and how much work is done at each level.
We can then solve the recurrence relations to determine an upper
bound to the worst case running time. 11
- Slide 13
- Example Consider the following simple example: 1. factorial(n)
2. if (n 1) 3. return(1) 4. else 5. return(n factorial(n-1) Let
T(n) be the running time of factorial(n): T(1) = a T(n) = T(n-1) +
b 12
- Slide 14
- Example Lets try to solve T(n): T(n) = T(n-1) + b = T(n-2) + 2b
= T(n-3) + 3b = T(n-(n-1)) + (n-1)b = a + (n-1)b = O(n) 13
- Slide 15
- More Example Consider the following: 1. SelectionSort(A[ ], i,
n) 2. if (i < n) 3. small = i 4. for (j = i + 1, j n, j++) 5. if
(A[j] < A[small]) 6. small = j 7. t = A[small] 8. A[small] =
A[i] 9. A[i] = t 10. SelectionSort(A, i + 1, n) 14
- Slide 16
- More Example Let T(m) denote the running time of
SelectSort(A,n-m+1,n). T(1) = 0 T(m) = T(m-1) + O(m) T(m-1) + C m
T(m-2) + C (m-1) + C m T(1) + C 2 + C 3 + + C m C (m+2)(m-1)/2 =
O(m 2 ) Therefore, the run time of SelectSort(A,1,n) is O(n 2 )
15
- Slide 17
- Class Discussion Consider MergeSort(): 1. MergeSort(A[ ]) 2. If
(length of A[ ] > 1) 3. Split A[ ] into two halves, A 1 [ ] and
A 2 [ ] 4. MergeSort(A 1 ) 5. MergeSort(A 2 ) 6. Merge A 1 and A 2
into one sorted list A What is the running time of MergeSort on an
array with n elements? 16 T(n) 2T(n/2) + Cn 2(2T(n/4) + C(n/2)) +
Cn 2lgnT(1) + lgn Cn = O(nlgn)
- Slide 18
- Master Theorem Let f be an increasing function that satisfies
the recurrence relation: f(n) = af(n/b) + cn d (assume n = b k ),
where k is a positive integer, a 1, b is an integer greater than 1,
c is a positive real number and d is a non-negative real number,
then: 17
- Slide 19
- Class Discussion Use the Master Theorem to find the Big-O of
MergeSort() 18
- Slide 20
- More Theorems Let f be a function that satisfies the recurrence
relation: f(n) = af(n-1) + bf(n-2) for n 2 and f(0) and f(1) have
given values. Let r 1 and r 2 be the roots of the equation: x 2 =
ax + b Then, if r 1 r 2, f(n) = c 1 r 1 n + c 2 r 2 n where n 0 for
some constants c 1 and c 2. If r 1 = r 2 = r, f(n) = (c 1 + c 2 n)r
n where n 0 for some constants c 1 and c 2. 19
- Slide 21
- Class Discussion Give a Big-O of the following recurse(): 1.
recurse(n) 2. if (n == 0) 3. return 1 4. for (i = 0; i < 4; i++)
5. total += recurse(n-2) 6. return total 20