Recurrences
description
Transcript of Recurrences
![Page 1: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/1.jpg)
RecurrencesDavid Kauchak
cs161Summer 2009
![Page 2: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/2.jpg)
Administrative Algorithms graded on efficiency!
Be specific about the run times (e.g. log bases)
Reminder: office hours today in Gates 195
![Page 3: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/3.jpg)
Recurrence A function that is defined with respect to itself
on smaller inputs
nnTnT )2/(2)(
nnTnT )4/(16)(
2)1(2)( nnTnT
![Page 4: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/4.jpg)
Why are we interested in recurrences? Computational cost of divide and conquer
algorithms
a subproblems of size n/b D(n) the cost of dividing the data C(n) the cost of recombining the subproblem solutions
In general, the runtimes of most recursive algorithms can be expressed as recurrences
)()()/()( nCnDbnaTnT
![Page 5: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/5.jpg)
The challenge Recurrences are often easy to define
because they mimic the structure of the program
But… they do not directly express the computational cost, i.e. n, n2, …
We want to remove self-recurrence and find a more understandable form for the function
![Page 6: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/6.jpg)
Three approaches Substitution method: when we have a good
guess of the solution, we prove that it’s correct
Recursion-tree method: If we don’t have a good guess of the solution, the recursion tree can help. Then solve with substitution method.
Master method: Provides solutions for recurrences of the form:
)()/()( nfbnaTnT
![Page 7: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/7.jpg)
Substitution method Guess the form of the solution Then prove it’s correct by induction
Halves the input and does a constant
dnTnT )2/()(
Guess?
![Page 8: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/8.jpg)
Substitution method Guess the form of the solution Then prove it’s correct by induction
Halves the input and does a constant Similar to binary search:
dnTnT )2/()(
Guess: O(log2 n)
![Page 9: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/9.jpg)
Assume T(k) = O(log2 k) for all k < n Show that T(n) = O(log2 n)
Given that T(n/2) = O(log2 n), then
T(n/2) ≤ c log2(n/2)
dnTnT )2/()(
0 allfor )()(0such that and constants positive exists there
:)())((nnncgnf
ncnfngO
![Page 10: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/10.jpg)
To prove that T(n) = O(log2 n) we need to identify the appropriate constants:
dnTnT )2/()(
dnTnT )2/()(dnc )2/(log2
dcnc 2loglog 22
dcnc 2log
nc 2log
if c ≥ d
0 allfor )()(0such that and constants positive exists there
:)())((nnncgnf
ncnfngO
i.e. some constant c such that T(n) ≤ c log2 n
residual
![Page 11: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/11.jpg)
Base case? For an inductive proof we need to show two
things: Assuming it’s true for k < n show it’s true for n Show that it holds for some base case
What is the base case in our situation?
otherwise)2/(
small is if)(
dnTn
nT
![Page 12: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/12.jpg)
Guess the solution? At each iteration, does a linear amount of work
(i.e. iterate over the data) and reduces the size by one at each step
O(n2)
Assume T(k) = O(k2) for all k < n again, this implies that T(n-1) = c(n-1)2
Show that T(n) = O(n2), i.e. T(n) ≤ cn2
nnTnT )1()(
![Page 13: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/13.jpg)
nnTnT )1()(nnc 2)1(
nnnc )12( 2
nccncn 22
2cn
if
residual
02 nccnnccn 2nnc )12(
12 nnc
nc
/121
which holds for any
c ≥1 for n ≥1
![Page 14: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/14.jpg)
Guess the solution? Recurses into 2 sub-problems that are half the
size and performs some operation on all the elements
O(n log n) What if we guess wrong, e.g. O(n2)?
Assume T(k) = O(k2) for all k < n again, this implies that T(n/2) = c(n/2)2
Show that T(n) = O(n2)
nnTnT )2/(2)(
![Page 15: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/15.jpg)
nnTnT )2/(2)(
nnc 2)2/(2ncn 4/2 2
ncn 22/1)2/1( 22 ncncn
2cnresidual
if
0)2/1( 2 ncn
02/1 2 ncn2cn
overkill?
![Page 16: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/16.jpg)
What if we guess wrong, e.g. O(n)?
Assume T(k) = O(k) for all k < n again, this implies that T(n/2) ≤ c
Show that T(n) = O(n)
nnTnT )2/(2)(
ncn 2/2ncn
nnTnT )2/(2)(
cnfactor of n so we can just roll it in?
![Page 17: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/17.jpg)
What if we guess wrong, e.g. O(n)?
Assume T(k) = O(k) for all k < n again, this implies that T(n/2) = c(n/2)
Show that T(n) = O(n)
nnTnT )2/(2)(
ncn 2/2ncn
nnTnT )2/(2)(
cnfactor of n so we can just roll it in?
Must prove the exact form!
cn+n ≤ cn ??
![Page 18: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/18.jpg)
Prove T(n) = O(n log2 n) Assume T(k) = O(k log2 k) for all k < n
again, this implies that T(k) = ck log2k Show that T(n) = O(n log2 n)
nnTnT )2/(2)(nncn )2/log(2/2
nnTnT )2/(2)(
nncn )2log(log 22
ncnncn 2log residualncn 2log
if cn ≥ n, c > 1
![Page 19: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/19.jpg)
Changing variables
Guesses? We can do a variable change: let m = log n
(or n = 2m)
Now, let S(m)=T(2m)
nnTnT log)(2)(
mTT mm )2(2)2( 2/
mmSmS )2/(2)(
![Page 20: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/20.jpg)
Changing variables
Guess?
mmTmS )2/(2)(
)log()( mmOmS
)log()()2()( mmOmSTnT m
)loglog(log)( nnOnT
substituting m=log n
![Page 21: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/21.jpg)
Recursion Tree
Guessing the answer can be difficult
The recursion tree approach Draw out the cost of the tree at each level of recursion Sum up the cost of the levels of the tree
Find the cost of each level with respect to the depth Figure out the depth of the tree Figure out (or bound) the number of leaves
Verify your answer using the substitution method
2)4/(3)( nnTnT
cnnTnTnT )3/2(2)3/()(
![Page 22: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/22.jpg)
2)4/(3)( nnTnT
cn2
T(n/4 ) T(n/4 )T(n/4 )
cn2
cost
![Page 23: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/23.jpg)
2)4/(3)( nnTnT
cn2 cn2
cost
2
4
nc
2
4
nc
2
4
nc
16nT
16nT
16nT
16nT
16nT
16nT
16nT
16nT
16nT
3/16cn2
![Page 24: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/24.jpg)
2)4/(3)( nnTnT
cn2 cn2
cost
2
4
nc
2
4
nc
2
4
nc
2
16
nc
3/16cn2
2
16
nc
2
16
nc
2
16
nc
2
16
nc
2
16
nc
2
16
nc
2
16
nc
2
16
nc (3/16)2cn2
What is the cost at each level? 2
163 cn
d
![Page 25: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/25.jpg)
What is the depth of the tree? At each level, the size of the data is divided
by 4
14
d
n
04
log
d
n
04loglog dn
nd log4log
nd 4log
![Page 26: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/26.jpg)
2)4/(3)( nnTnT
cn2
2
4
nc
2
4
nc
2
4
nc
2
16
nc
2
16
nc
2
16
nc
2
16
nc
2
16
nc
2
16
nc
2
16
nc
2
16
nc
2
16
nc
How many leaves are there?
)1(T
![Page 27: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/27.jpg)
How many leaves? How many leaves are there in a complete
ternary tree of depth d?
nd 4log33
![Page 28: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/28.jpg)
Total cost
)3(163...
163
163)( 4log2
12
222 n
d
cncncncnnT
)3(163
44
log1log
0
2 nn
i
i
cn
xx
kk
11
0
let x = 3/16
)3(163
4log
0
2 n
i
i
cn
)3()16/3(1
14log2 ncn
)3(1316
4log2 ncn ?
![Page 29: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/29.jpg)
Total cost )3(1316)( 4log2 ncnnT
nn 4log44 3loglog 43
3loglog 444 n34log
4log4 n3log4n
)(1316)( 3log2 4ncnnT
)()( 2nOnT
![Page 30: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/30.jpg)
Verify solution using substitution
Assume T(k) = O(k2) for all k < n Show that T(n) = O(n2)
Given that T(n/4) = O((n/4)2), then
T(n/4) ≤ c(n/4)2
2)4/(3)( nnTnT
0 allfor )()(0such that and constants positive exists there
:)())((nnncgnf
ncnfngO
![Page 31: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/31.jpg)
To prove that Show that T(n) = O(n2) we need to identify the appropriate constants:
2)4/(3)( nnTnT 22)4/(3 nnc
2cn
if
0 allfor )()(0such that and constants positive exists there
:)())((nnncgnf
ncnfngO
i.e. some constant c such that T(n) ≤ cn2
2)4/(3)( nnTnT
22 16/3 ncn
1316
c
![Page 32: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/32.jpg)
Master Method Provides solutions to the recurrences of the form:
)()/()( nfbnaTnT
)()( then ,0for )()( if loglog aa bb nnTnOnf
)log()( then ),()( if loglog nnnTnnf aa bb
1for )()/( and 0for )()( if log cncfbnafnnf ab
))(()(then nfnT
![Page 33: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/33.jpg)
nnTnT )4/(16)(
a = b =f(n) =
164n
abnlog
2
16log4
nn
?)( is?)( is
?)( is
2
2
2
nnnnnOn
)()( then ,0for )()( if loglog aa bb nnTnOnf
)log()( then ),()( if loglog nnnTnnf aa bb 1for )()/( and 0for )()( if log cncfbnafnnf ab
))(()(then nfnT
Case 1: Θ(n2)
![Page 34: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/34.jpg)
nnTnT 2)2/()(
a = b =f(n) =
122n
abnlog
0
1log2
nn
?)(2 is?)(2 is
?)(2 is
0
0
0
nnnO
n
n
n
)()( then ,0for )()( if loglog aa bb nnTnOnf
)log()( then ),()( if loglog nnnTnnf aa bb 1for )()/( and 0for )()( if log cncfbnafnnf ab
))(()(then nfnT
Case 3? ?1for 22 is 2/ cc nn
![Page 35: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/35.jpg)
nnTnT 2)2/()( )()( then ,0for )()( if loglog aa bb nnTnOnf
)log()( then ),()( if loglog nnnTnnf aa bb 1for )()/( and 0for )()( if log cncfbnafnnf ab
))(()(then nfnT
T(n) = Θ(2n)
?1for 22 is 2/ cc nn
Let c = 1/2
nn 2)2/1(2 2/ nn 222 12/
12/ 22 nn
![Page 36: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/36.jpg)
nnTnT )2/(2)(
a = b =f(n) =
22n
abnlog
1
2log2
nn
?)( is?)( is
?)( is
1
1
1
nnnnnOn
)()( then ,0for )()( if loglog aa bb nnTnOnf
)log()( then ),()( if loglog nnnTnnf aa bb 1for )()/( and 0for )()( if log cncfbnafnnf ab
))(()(then nfnT
Case 2: Θ(n log n)
![Page 37: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/37.jpg)
!)4/(16)( nnTnT
a = b =f(n) =
164n!
abnlog
2
16log4
nn
?)(! is?)(! is
?)(! is
2
2
2
nnnnnOn
)()( then ,0for )()( if loglog aa bb nnTnOnf
)log()( then ),()( if loglog nnnTnnf aa bb 1for )()/( and 0for )()( if log cncfbnafnnf ab
))(()(then nfnT
Case 3? ?1for 416 is ccn!)!(n/
![Page 38: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/38.jpg)
!)4/(16)( nnTnT )()( then ,0for )()( if loglog aa bb nnTnOnf
)log()( then ),()( if loglog nnnTnnf aa bb 1for )()/( and 0for )()( if log cncfbnafnnf ab
))(()(then nfnT
T(n) = Θ(n!) Let c = 1/2
?1for 416 is ccn!)!(n/
!2/1! ncn )!2/(n
!2/1)!2/()!4/(16 nnn
therefore,
![Page 39: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/39.jpg)
nnTnT log)2/(2)(
a = b =f(n) =
2logn
abnlog
nnn
2/12
2
2log
2log
?)(log is?)(log is
?)(log is
2/1
2/1
2/1
nnnnnOn
)()( then ,0for )()( if loglog aa bb nnTnOnf
)log()( then ),()( if loglog nnnTnnf aa bb 1for )()/( and 0for )()( if log cncfbnafnnf ab
))(()(then nfnT
Case 1: Θ( )
2
n
![Page 40: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/40.jpg)
nnTnT )2/(4)(
a = b =f(n) =
42n
abnlog
2
4log2
nn
?)( is?)( is
?)( is
2
2
2
nnnnnOn
)()( then ,0for )()( if loglog aa bb nnTnOnf
)log()( then ),()( if loglog nnnTnnf aa bb 1for )()/( and 0for )()( if log cncfbnafnnf ab
))(()(then nfnT
Case 1: Θ(n2)
![Page 41: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/41.jpg)
Why does the mastermethod work?
)()/()( nfbnaTnT
)(nf
a
)/( bnf )/( bnf )/( bnf
)/( 2bnf
a
)/( 2bnf )/( 2bnf )/( 2bnf
a
)/( 2bnf )/( 2bnf )/( 2bnf
a
)/( 2bnf )/( 2bnf
)(nf
)/( bnaf
)/( 22 bnfa
![Page 42: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/42.jpg)
What is the depth of the tree? At each level, the size of the data is divided
by b
1dbn
0log
dbn
04loglog bn
nbd loglog
nd blog
![Page 43: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/43.jpg)
How many leaves? How many leaves are there in a complete
a-ary tree of depth d?
nd baa logabnlog
![Page 44: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/44.jpg)
Total cost
)()/(...)/()/()()( 3log1122 ann bnbnfabnfabnafncfnT
)()/( log1log
0
an
i
ii bb
nbnfa
)()( then ,0for )()( if loglog aa bb nnTnOnf
)log()( then ),()( if loglog nnnTnnf aa bb 1for )()/( and 0for )()( if log cncfbnafnnf ab
))(()(then nfnT
Case 1: cost is dominated by the cost of the leaves
)()/( log1log
0
an
i
ii bb
nbnfa
![Page 45: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/45.jpg)
Total cost
)()/(...)/()/()()( 3log1122 ann bnbnfabnfabnafncfnT
)()/( log1log
0
an
i
ii bb
nbnfa
)()( then ,0for )()( if loglog aa bb nnTnOnf
)log()( then ),()( if loglog nnnTnnf aa bb 1for )()/( and 0for )()( if log cncfbnafnnf ab
))(()(then nfnT
Case 2: cost is evenly distributed across tree
As we saw with mergesort, log n levels to the tree and at each level f(n) work
![Page 46: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/46.jpg)
Total cost
)()/(...)/()/()()( 3log1122 ann bnbnfabnfabnafncfnT
)()/( log1log
0
an
i
ii bb
nbnfa
)()( then ,0for )()( if loglog aa bb nnTnOnf
)log()( then ),()( if loglog nnnTnnf aa bb 1for )()/( and 0for )()( if log cncfbnafnnf ab
))(()(then nfnT
Case 3: cost is dominated by the cost of the root
)(nf
a
![Page 47: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/47.jpg)
Don’t shoot the messenger Why do we care about substitution method
and recurrence tree method? Master method is much easier.
Some recurrences don’t fit the mold
cnnTnTnT )3/2(2)3/()(
![Page 48: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/48.jpg)
Other forms of the master method
adnOadnnOadnO
nT
ba
bd
bd
b log if)(log if)log(log if)(
)(log
)()/()( dnObnaTnT
![Page 49: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/49.jpg)
Recurrences
dnTnT )3/(2)(
nnTnT log)1()(
nnTnT )7/(7)(
3)2/(8)( nnTnT
)()( then ,0for )()( if loglog aa bb nnTnOnf
)log()( then ),()( if loglog nnnTnnf aa bb
))(()(then nfnT 1for )()/( and 0for )()( if log cncfbnafnnf ab
![Page 50: Recurrences](https://reader036.fdocuments.in/reader036/viewer/2022081604/56815444550346895dc24f3b/html5/thumbnails/50.jpg)
Practice problem You are given an infinitely long array, where
the first n elements contain data and the remaining elements do not. At any given position, you can test whether that element contains data or not. How can you determine n?