Analysis of Recursive Algorithms October 29, 2014 1
Slide 3
Recursive Algorithms Analysis becomes a bit more complicated
for recursive algorithms. Recurrence relations is helpful for such
analyses. A recursive step can be expressed as a recurrence
relation, showing how earlier terms in a sequence are related to
later terms. 2
Slide 4
Recurrence Relation A recurrence relation for a sequence a 1, a
2, a 3, is a formula that relates each term a k to its certain
predecessors a k-1, a k-2, , a k-i where i is a fixed integer and k
is any integer greater than i. The initial conditions for such a
recurrence relation specify the values of a 1, a 2, , a i-1. 3
Slide 5
Example For the Tower of Hanoi problem: F 1 = 1 F n = 2F n-1 +
1 (It can be shown with a complex proof that 2 n -1 is the optimal
number of steps.) Fibonacci Sequence: F 0 = 0 F 1 = 1 F n = F n-1 +
F n-2 4
Slide 6
Class Discussion Can you set up some recurrence relations to
compute f(n) where f(n) is the number of 0-1 bit strings with n
bits and with no consecutive 1s, i.e., no 11 patterns? Can you set
up a recurrence relation to count the number of ways to
parenthesize x 1 x 2 x n ? E.g., when n = 3, there are two ways,
((x 1 x 2 ) x 3 ) and (x 1 (x 2 x 3 )). 5 f(n) = f 0 (n)+f 1 (n); f
0 (n) = f(n-1); f 1 (n) = f 0 (n-1); f 0 (1) = f 1 (1) = 1
Slide 7
Repeated Substitution Solving recurrence relation is the key to
analyze recursive algorithms. One direct method of solving
recurrence relations is called repeated substitution. This is done
by repeatedly substitute the formulae for the earlier terms in the
recurrence relation. 6
Slide 8
Repeated Substitution For example: a 0 = 1 a k = a k-1 + 2 We
start by: a k = a k-1 + 2 = a k-2 + 2 + 2 = a k-3 + 2 + 2 + 2 = a 0
+ 2k = 1 + 2k 7
Slide 9
Induction We can also start with a guess of the closed form
solution. Then, prove its correctness with induction. 8
Slide 10
Induction For example: a 0 = 1 a k = a k-1 + 2 Suppose we want
to show that a k =1+2k: Base case: k = 0, a 0 = 1 = 1 + 2 0
Inductive case: Assume that a k = 1+2k for k 0 a k+1 = a k + 2 = 1
+ 2k + 2 = 1 + 2(k+1) Therefore, we can conclude that a k =1+2k
(Structural Induction) 9
Slide 11
Class Discussion Give a closed form solution to the following
recurrence relation: a 0 = 5 a n = na n-1 10
Slide 12
Recursive Algorithms Solving recurrence relations is the key to
analyze recursive algorithms. A recurrence relation can be used to
express the running time of a recursive algorithm, showing how many
recursive calls are made and how much work is done at each level.
We can then solve the recurrence relations to determine an upper
bound to the worst case running time. 11
Slide 13
Example Consider the following simple example: 1. factorial(n)
2. if (n 1) 3. return(1) 4. else 5. return(n factorial(n-1) Let
T(n) be the running time of factorial(n): T(1) = a T(n) = T(n-1) +
b 12
Slide 14
Example Lets try to solve T(n): T(n) = T(n-1) + b = T(n-2) + 2b
= T(n-3) + 3b = T(n-(n-1)) + (n-1)b = a + (n-1)b = O(n) 13
Slide 15
More Example Consider the following: 1. SelectionSort(A[ ], i,
n) 2. if (i < n) 3. small = i 4. for (j = i + 1, j n, j++) 5. if
(A[j] < A[small]) 6. small = j 7. t = A[small] 8. A[small] =
A[i] 9. A[i] = t 10. SelectionSort(A, i + 1, n) 14
Slide 16
More Example Let T(m) denote the running time of
SelectSort(A,n-m+1,n). T(1) = 0 T(m) = T(m-1) + O(m) T(m-1) + C m
T(m-2) + C (m-1) + C m T(1) + C 2 + C 3 + + C m C (m+2)(m-1)/2 =
O(m 2 ) Therefore, the run time of SelectSort(A,1,n) is O(n 2 )
15
Slide 17
Class Discussion Consider MergeSort(): 1. MergeSort(A[ ]) 2. If
(length of A[ ] > 1) 3. Split A[ ] into two halves, A 1 [ ] and
A 2 [ ] 4. MergeSort(A 1 ) 5. MergeSort(A 2 ) 6. Merge A 1 and A 2
into one sorted list A What is the running time of MergeSort on an
array with n elements? 16 T(n) 2T(n/2) + Cn 2(2T(n/4) + C(n/2)) +
Cn 2lgnT(1) + lgn Cn = O(nlgn)
Slide 18
Master Theorem Let f be an increasing function that satisfies
the recurrence relation: f(n) = af(n/b) + cn d (assume n = b k ),
where k is a positive integer, a 1, b is an integer greater than 1,
c is a positive real number and d is a non-negative real number,
then: 17
Slide 19
Class Discussion Use the Master Theorem to find the Big-O of
MergeSort() 18
Slide 20
More Theorems Let f be a function that satisfies the recurrence
relation: f(n) = af(n-1) + bf(n-2) for n 2 and f(0) and f(1) have
given values. Let r 1 and r 2 be the roots of the equation: x 2 =
ax + b Then, if r 1 r 2, f(n) = c 1 r 1 n + c 2 r 2 n where n 0 for
some constants c 1 and c 2. If r 1 = r 2 = r, f(n) = (c 1 + c 2 n)r
n where n 0 for some constants c 1 and c 2. 19
Slide 21
Class Discussion Give a Big-O of the following recurse(): 1.
recurse(n) 2. if (n == 0) 3. return 1 4. for (i = 0; i < 4; i++)
5. total += recurse(n-2) 6. return total 20