RAIK 283: Data Structures & Algorithms
description
Transcript of RAIK 283: Data Structures & Algorithms
![Page 1: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/1.jpg)
1
Recursive Algorithm Analysis
Dr. Ying [email protected]
RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms
September 13, 2012
![Page 2: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/2.jpg)
2
Giving credit where credit is due:Giving credit where credit is due:• Most of the lecture notes are based on the slides Most of the lecture notes are based on the slides
from the Textbook’s companion websitefrom the Textbook’s companion websitehttp://www.aw-bc.com/info/levitinhttp://www.aw-bc.com/info/levitin
• Several slides are from Hsu Wen Jing of the Several slides are from Hsu Wen Jing of the National University of SingaporeNational University of Singapore
• I have modified them and added new slidesI have modified them and added new slides
RAIK 283: Data Structures & RAIK 283: Data Structures & AlgorithmsAlgorithms
![Page 3: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/3.jpg)
3
Example: a recursive algorithmExample: a recursive algorithm
Algorithm:Algorithm:if if nn=0 then =0 then FF((nn) := 1) := 1
else else FF((nn) := ) := FF((nn-1) * -1) * nnreturn return FF((nn) )
![Page 4: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/4.jpg)
4
Algorithm F(n)
// Compute the nth Fibonacci number recursively
//Input: A nonnegative integer n
//Output: the nth Fibonacci number
if n 1 return n
else return F(n-1) + F(n-2)
Example: another recursive Example: another recursive algorithmalgorithm
![Page 5: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/5.jpg)
5
Recurrence RelationRecurrence Relation Recurrence RelationRecurrence Relation
![Page 6: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/6.jpg)
6
Recurrence RelationRecurrence Relation Recurrence Relation:Recurrence Relation: an equation or inequality that an equation or inequality that
describes a function in terms of its value on smaller describes a function in terms of its value on smaller inputsinputs
![Page 7: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/7.jpg)
7
Example: a recursive algorithmExample: a recursive algorithm Algorithm:Algorithm:
if if nn=0 then =0 then FF((nn) := 1) := 1else else FF((nn) := ) := FF((nn-1) * -1) * nn
return return FF((nn) )
What does this algorithm compute?What does this algorithm compute?
![Page 8: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/8.jpg)
8
Example: a recursive algorithmExample: a recursive algorithm Algorithm:Algorithm:
if if nn=0 then =0 then FF((nn) := 1) := 1else else FF((nn) := ) := FF((nn-1) * -1) * nn
return return FF((nn) )
What does this algorithm compute?What does this algorithm compute?
What’s the basic operation of this algorithm?What’s the basic operation of this algorithm?
![Page 9: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/9.jpg)
9
Example: recursive evaluation Example: recursive evaluation of of nn ! ! Recursive definition of Recursive definition of nn!:!: Algorithm:Algorithm:
if if nn=0 then =0 then FF((nn) := 1) := 1else else FF((nn) := ) := FF((nn-1) * -1) * nn
return return FF((nn) )
M(n): number of multiplications to compute M(n): number of multiplications to compute nn! ! with this recursive algorithmwith this recursive algorithm
![Page 10: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/10.jpg)
10
Example: recursive evaluation Example: recursive evaluation of of nn ! ! Recursive definition of Recursive definition of nn!:!: Algorithm:Algorithm:
if if nn=0 then =0 then FF((nn) := 1) := 1else else FF((nn) := ) := FF((nn-1) * -1) * nn
return return FF((nn) )
M(n): number of multiplications to compute M(n): number of multiplications to compute nn! ! with this recursive algorithmwith this recursive algorithm
Could we establish a Could we establish a recurrence relationrecurrence relation for for deriving M(n)? deriving M(n)?
![Page 11: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/11.jpg)
11
Example: recursive evaluation Example: recursive evaluation of of nn ! ! Definition:Definition: n n ! = 1*2! = 1*2*…*(n-*…*(n-1)*1)*nn Recursive definition of Recursive definition of nn!:!: Algorithm:Algorithm:
if if nn=0 then =0 then FF((nn) := 1) := 1else else FF((nn) := ) := FF((nn-1) * -1) * nn
return return FF((nn) )
M(n) = M(n-1) + 1M(n) = M(n-1) + 1 Initial Condition: M(0) = ?Initial Condition: M(0) = ?
![Page 12: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/12.jpg)
12
Example: recursive evaluation Example: recursive evaluation of of nn ! ! Recursive definition of Recursive definition of nn!:!: Algorithm:Algorithm:
if if nn=0 then =0 then FF((nn) := 1) := 1else else FF((nn) := ) := FF((nn-1) * -1) * nn
return return FF((nn) )
M(n) = M(n-1) + 1M(n) = M(n-1) + 1 Initial condition: M(0) = 0Initial condition: M(0) = 0 Explicit formula for M(n) in terms of n only? Explicit formula for M(n) in terms of n only?
![Page 13: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/13.jpg)
13
Time efficiency of recursive Time efficiency of recursive algorithmsalgorithmsSteps in analysis of recursive algorithms: Steps in analysis of recursive algorithms: Decide on parameter Decide on parameter nn indicating indicating input sizeinput size Identify algorithm’s Identify algorithm’s basic operationbasic operation Determine Determine worstworst, , averageaverage, and , and bestbest case for inputs of size case for inputs of size nn
Set up a recurrence relation and initial condition(s) for Set up a recurrence relation and initial condition(s) for CC((nn)-the number of times the basic operation will be )-the number of times the basic operation will be executed for an input of size executed for an input of size nn
Solve the recurrence to obtain a closed form or Solve the recurrence to obtain a closed form or determine the order of growth of the solution (see determine the order of growth of the solution (see Appendix B)Appendix B)
![Page 14: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/14.jpg)
14
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi Problem:Problem:
• Given three pegs (A, B, C) and n disks of different sizes Given three pegs (A, B, C) and n disks of different sizes • Initially, all the disks are on peg A in order of size, the largest on Initially, all the disks are on peg A in order of size, the largest on
the bottom and the smallest on top the bottom and the smallest on top • The goal is to move all the disks to peg C using peg B as an The goal is to move all the disks to peg C using peg B as an
auxiliaryauxiliary• Only 1 disk can be moved at a time, and a larger disk cannot be Only 1 disk can be moved at a time, and a larger disk cannot be
placed on top of a smaller oneplaced on top of a smaller one
A C
B
n disks
![Page 15: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/15.jpg)
15
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi Design a recursive algorithm to solve this problem:Design a recursive algorithm to solve this problem:
• Given three pegs (A, B, C) and n disks of different sizes Given three pegs (A, B, C) and n disks of different sizes • Initially, all the disks are on peg A in order of size, the largest on Initially, all the disks are on peg A in order of size, the largest on
the bottom and the smallest on top the bottom and the smallest on top • The goal is to move all the disks to peg C using peg B as an The goal is to move all the disks to peg C using peg B as an
auxiliaryauxiliary• Only 1 disk can be moved at a time, and a larger disk cannot be Only 1 disk can be moved at a time, and a larger disk cannot be
placed on top of a smaller oneplaced on top of a smaller one
A C
B
n disks
![Page 16: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/16.jpg)
16
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
Step 1: Solve simple case when n<=1?Step 1: Solve simple case when n<=1?Just trivialJust trivial
A C
B
A C
B
Move(A, C)
![Page 17: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/17.jpg)
17
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
Step 2: Assume that a smaller instance can be Step 2: Assume that a smaller instance can be solved, i.e. can move n-1 disks. Then?solved, i.e. can move n-1 disks. Then?
A C
B
A C
B
A C
B
![Page 18: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/18.jpg)
18
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
A C
BA C
B
A C
B
A C
B
![Page 19: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/19.jpg)
19
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
A C
BA C
B
A C
B
A C
B
TOWER(n, A, B, C)
![Page 20: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/20.jpg)
20
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
A C
BA C
B
A C
B
A C
B
TOWER(n, A, B, C)
TOWER(n-1, A, C, B)
Move(A, C)
TOWER(n-1, B, A, C)
![Page 21: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/21.jpg)
21
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
TOWER(n, A, B, C) {
TOWER(n-1, A, C, B); Move(A, C);TOWER(n-1, B, A, C)}
if n<1 return;
![Page 22: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/22.jpg)
22
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
TOWER(n, A, B, C) {
TOWER(n-1, A, C, B); Move(A, C);TOWER(n-1, B, A, C)}
if n<1 return;
Algorithm analysis: Algorithm analysis: Input size? Basic operation?Input size? Basic operation?
![Page 23: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/23.jpg)
23
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
TOWER(n, A, B, C) {
TOWER(n-1, A, C, B); Move(A, C);TOWER(n-1, B, A, C)}
if n<1 return;
Algorithm analysis: Algorithm analysis: Do we need to differentiate best case, worst Do we need to differentiate best case, worst
case & average case for inputs of size n?case & average case for inputs of size n?
![Page 24: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/24.jpg)
24
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
TOWER(n, A, B, C) {
TOWER(n-1, A, C, B); Move(A, C);TOWER(n-1, B, A, C)}
if n<1 return;
Algorithm analysis: Algorithm analysis: Set up a recurrence relation and initial Set up a recurrence relation and initial
condition(s) for C(n)condition(s) for C(n)
![Page 25: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/25.jpg)
25
EXAMPLE: tower of hanoiEXAMPLE: tower of hanoi
TOWER(n, A, B, C) {
TOWER(n-1, A, C, B); Move(A, C);TOWER(n-1, B, A, C)}
if n<1 return;
Algorithm analysis: Algorithm analysis: C(n) = 2C(n-1)+1C(n) = 2C(n-1)+1
![Page 26: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/26.jpg)
26
In-Class ExerciseIn-Class Exercise P. 76 Problem 2.4.1 (c): solve this recurrence relation: P. 76 Problem 2.4.1 (c): solve this recurrence relation:
x(n) = x(n-1) + n for n>0, x(0)=0x(n) = x(n-1) + n for n>0, x(0)=0 P. 77 Problem 2.4.4: consider the following recursive algorithm: P. 77 Problem 2.4.4: consider the following recursive algorithm:
• Algorithm Q(n)Algorithm Q(n)// Input: A positive integer n// Input: A positive integer nIf n = 1 return 1If n = 1 return 1else return Q(n-1) + 2 * n – 1else return Q(n-1) + 2 * n – 1
• A. Set up a recurrence relation for this function’s values and solve it A. Set up a recurrence relation for this function’s values and solve it to determine what this algorithm computesto determine what this algorithm computes
• B. Set up a recurrence relation for the number of multiplications B. Set up a recurrence relation for the number of multiplications made by this algorithm and solve it. made by this algorithm and solve it.
• C. Set up a recurrence relation for the number of C. Set up a recurrence relation for the number of additions/subtractions made by this algorithm and solve it. additions/subtractions made by this algorithm and solve it.
![Page 27: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/27.jpg)
27
Example: BinRec(n) Example: BinRec(n)
Algorithm BinRec(n)
//Input: A positive decimal integer n
//Output: The number of binary digits in n’s binary representation
if n = 1 return 1
else return BinRec( n/2 ) + 1
![Page 28: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/28.jpg)
28
Smoothness ruleSmoothness ruleIf T(n) If T(n) (f(n)) (f(n)) for values of n that are powers of b, where b for values of n that are powers of b, where b 2, 2,
then then
T(n) T(n) (f(n))(f(n))
![Page 29: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/29.jpg)
29
Example: BinRec(n) Example: BinRec(n)
Algorithm BinRec(n)
//Input: A positive decimal integer n
//Output: The number of binary digits in n’s binary representation
if n = 1 return 1
else return BinRec( n/2 ) + 1
If C(n) If C(n) (f(n)) (f(n)) for values of n that are powers of b, where b for values of n that are powers of b, where b 2, 2, then then C(n) C(n) (f(n))(f(n))
![Page 30: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/30.jpg)
30
Fibonacci numbersFibonacci numbers The Fibonacci sequence:The Fibonacci sequence:
0, 1, 1, 2, 3, 5, 8, 13, 21, … 0, 1, 1, 2, 3, 5, 8, 13, 21, …
Fibonacci recurrence:Fibonacci recurrence:F(F(nn) = F() = F(nn-1) + F(-1) + F(nn-2) -2) F(0) = 0 F(0) = 0 F(1) = 1F(1) = 1
2nd 2nd order linear homogeneous order linear homogeneous recurrence relation recurrence relation
with constant coefficientswith constant coefficients
![Page 31: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/31.jpg)
31
Solving linear homogeneous Solving linear homogeneous recurrence relations with constant recurrence relations with constant coefficientscoefficients
Easy first: 1Easy first: 1stst order LHRRCCs: order LHRRCCs:CC((nn) = ) = a Ca C((nn -1) -1) CC(0) = (0) = tt … Solution: … Solution: CC((nn) =) = t a t ann
Extrapolate to 2Extrapolate to 2ndnd order orderLL((nn) = ) = aa LL((nn-1) + -1) + bb LL((nn-2) … A solution?: -2) … A solution?: LL((nn) = ) = ??
Characteristic equation (quadratic)Characteristic equation (quadratic) Solve to obtain roots Solve to obtain roots rr11 and and rr2 2 ((quadratic formula)) General solution to RR: linear combination of General solution to RR: linear combination of rr11
nn and and rr22
nn
Particular solution: use initial conditionsParticular solution: use initial conditions
![Page 32: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/32.jpg)
32
Solving linear homogeneous Solving linear homogeneous recurrence relations with constant recurrence relations with constant coefficientscoefficients
Easy first: 1Easy first: 1stst order LHRRCCs: order LHRRCCs:CC((nn) = ) = a Ca C((nn -1) -1) CC(0) = (0) = tt … Solution: … Solution: CC((nn) =) = t a t ann
Extrapolate to 2Extrapolate to 2ndnd order orderLL((nn) = ) = aa LL((nn-1) + -1) + bb LL((nn-2) … A solution?: -2) … A solution?: LL((nn) = ) = ??
Characteristic equation (quadratic)Characteristic equation (quadratic) Solve to obtain roots Solve to obtain roots rr11 and and rr2 2 ((quadratic formula)) General solution to RR: linear combination of General solution to RR: linear combination of rr11
nn and and rr22
nn
Particular solution: use initial conditionsParticular solution: use initial conditions
Explicit Formula for Explicit Formula for Fibonacci Number: F(n) = F(n-1) +F(n-2)Fibonacci Number: F(n) = F(n-1) +F(n-2)
![Page 33: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/33.jpg)
33
1. Definition based recursive algorithm1. Definition based recursive algorithm
Computing Fibonacci numbersComputing Fibonacci numbers
Algorithm F(n)
// Compute the nth Fibonacci number recursively
//Input: A nonnegative integer n
//Output: the nth Fibonacci number
if n 1 return n
else return F(n-1) + F(n-2)
![Page 34: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/34.jpg)
34
2. Nonrecursive brute-force algorithm2. Nonrecursive brute-force algorithm
Computing Fibonacci numbersComputing Fibonacci numbers
Algorithm Fib(n)
// Compute the nth Fibonacci number iteratively
//Input: A nonnegative integer n
//Output: the nth Fibonacci number
F[0] 0; F[1] 1
for i 2 to n do
F[i] F[i-1] + F[i-2]
return F[n]
![Page 35: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/35.jpg)
35
Computing Fibonacci numbersComputing Fibonacci numbers
3. Explicit formula algorithm3. Explicit formula algorithm
integernearest the torounded )2
51(5
1)( nnF
Special care in its implementation: Special care in its implementation:
Intermediate results are irrational numbersIntermediate results are irrational numbersTheir approximations in the computer are accurate enoughTheir approximations in the computer are accurate enough
Final round-off yields a correct result Final round-off yields a correct result
![Page 36: RAIK 283: Data Structures & Algorithms](https://reader035.fdocuments.in/reader035/viewer/2022070500/5681681b550346895ddda811/html5/thumbnails/36.jpg)
36
In-Class Exercises In-Class Exercises What is the explicit formula for A(n)?What is the explicit formula for A(n)?
A(A(nn) = 3A() = 3A(nn-1) – 2A(-1) – 2A(nn-2) A(0) = 1 A(1) = 3-2) A(0) = 1 A(1) = 3
P.83 2.5.3. Climbing stairs: Find the number of P.83 2.5.3. Climbing stairs: Find the number of different ways to climb an n-stair stair-case if each different ways to climb an n-stair stair-case if each step is either one or two stairs. (For example, a 3-step is either one or two stairs. (For example, a 3-stair staircase can be climbed three ways: 1-1-1, 1-stair staircase can be climbed three ways: 1-1-1, 1-2, and 2-1.)2, and 2-1.)