Chapter 4: Solution of recurrence relationships
Techniques:Substitution: Guess answer, prove by induction
Tree analysis: Good way to develop a guess
Master theorem: Recipe when T(n) = aT(n\b) + f(n)with a >1, b >1 and f(n) a specified function
Example: use substitution method to find run time for merge sort
When n is a power of 2
The recurrence for merge sort (worst case) is
T(n) = 2 n=2T(n) = 2T(n/2) +n n=2k k>1
Proved by induction on integers (HW1)T(n) = nlgn= 2k lg2k = k2k
Worst case merge sort for any n
T(n) = constant n=1T(n) = T(floor(n/2)) + T(ceiling(n/2)) + Q(n) n>1
Look for order of growth instead of analytic solution
We can ignore floor and ceiling and replace Q(n) by cn with c > 0
Recurrence becomes T(n) = 2T(n/2) + cn
Prove T(n) = Q(nlgn) by substitution method
Based on Theorem 3.1, T(n) = Q(nlgn) iff T(n) = O(nlgn) and T(n) = W(nlgn)
Start by proving T(n) = O(nlgn) is a solution of T(n) = 2T(n/2) + cn
if -dn + cn = 0; therefore d = c > 0 will work as the constant in the definition of big O
For a more general solution start with the string on inequalities
T(n) < dnlg(n)-dn+cn < dnlg(n),
which says
if T(n) < dnlg(n)-dn+cn
and dnlg(n)-dn+cn < dnlg(n)
then T(n) < dnlg(n),
which is what we need in order to show that T(n)=O(nlg(n)).
But T(n) < dnlg(n)-dn+cn < dnlg(n)
also put a condition on d by the inequality
dnlg(n)-dn+cn < dnlg(n)
Solving this inequality for d yields
d > c > 0,
which says that d = c is OK as the constant in the definition of big O but any d bigger than c is also OK
Finish by proving T(n) = W(nlgn) is a solution ofT(n) = 2T(n/2) + cnAssume T(n/2)= (W (n/2)lg(n/2))
if -dn + cn = 0; therefore d = c > 0 will work as the constant in the definition of big W
For a more general solution start with the string on inequalities
T(n) > dnlg(n)-dn+cn > dnlg(n),
which says
if T(n) > dnlg(n)-dn+cn
and dnlg(n)-dn+cn > dnlg(n)
then T(n) > dnlg(n),
which is what we need in order to show that T(n)=W(nlg(n)).
But T(n) > dnlg(n)-dn+cn > dnlg(n)
also put a condition on d by the inequality
dnlg(n)-dn+cn > dnlg(n)
Solving this inequality for d yields
0 < d < c,
which says that d = c is OK as the constant in the definition of big W but any d between 0 and c is also OK
Sometimes, even with correct guess, substitution does not work To prove Big O in these cases, try subtracting a lower-order term in your assumption
Example: T(n) = 8T(n/2) + Q(n2)Show T(n) = O(n3)
if d = c > 0
The string of inequalities T(n) < dn3 + cn2 < dn3 cannot be true because cn2 > 0
The substitution T(n/2) = O((n/2)3 will not work in this case
Try “subtract off a lower order term”assume T(n/2) < d[(n/2)3 – (n/2)2]
((n/2)3)
if d = c > 0
if we assume T(n/2) < d[(n/2)3 – (n/2)2] then we most show T(n) < d[n3-n2]
< d[n3-n2] implies
T(n) < d[n3-n2] and 0 < c < d (i.e. any d > c satisfies the definition of big O)
Substitution T(n/2) = W((n/2)3) works for proving T(n) = W(n3)is a solution of T(n) = 8T(n/2) + cn2 with c > 0
Assuming T(n/2) = W((n/2)3) means there exist d>0 such thatT(n/2) > d(n/2)3 which impliesT(n) > 8d(n/2)3 + cn2
The string of inequalities T(n) > dn3 + cn2 > dn3 can be true because cn2 > 0,which means that any d > 0 is OK as the constant in the definition of big ;W hence T(n) = W(n3)
Quiz Friday, 2-6-15Evaluate and bound sums
prove formulas by induction on integersconvert one sum into anotherbound by inductionbound every termbound by integration
CptS 450 Spring 2015[All problems are from Cormen et al, 3rd Edition] Homework Assignment 4: due 2/11/151. ex 4.4-7 on page 93 (tree analysis and substitution)
2. problem 4-3f on page 108 by tree analysis and substitution
Tree analysisAt each node write overhead per recurrence
usually a function of level indexCost of leaves = number of leavesCalculate istopSum level cost for i = 0 to istop -1Add cost of leaves
If tree analysis is ambiguous, verify by substitution
Example: T(n) = 3T(|_n/4_|) + Q(n2) ~ 3T(n/4) + cn2
As indicated in previous slide, levels are indexed from zero
ISTOP is the index of leavesAt ISTOP divide and conquer has bottomed out with subgroups of size n0
Could evaluate cost of level using finite geometric sum
Bounding by infinite series easier
series
Test guess from tree analysis by substitution
Guess is upper bound onlyDid not evaluate sum over levels
= 1
solving right-most inequality for d yields d > (16/13)c > 0 for all n > 1there exists d > 0 such that 0 < T(n) < dn2 all n > 1hence, T(n) = O(n2)
< dn2
HW 4 due date changed
Homework Assignment 4: due 2/13/151. ex 4.4-7 on page 93 (tree analysis and substitution)
2. problem 4-3f on page 108 by tree analysis and substitution
Quiz Friday 2/13Substitution method
set up (remove floors, ceiling, asymptotic notation)assumptionimplementation of assumptionconstants and n0
Imbalanced tree analysis continued
Test guess by substitution
< 0 for d
= all n > 0
< dnlgn
solve
d > c/(lg3-2/3) > 0
substitution
= 0
< cn3
T(n) < cn3 - 6cn2 + 12cn -8c + n2 < cn3
Solve - 6cn2 + 12cn -8c + n2 < 0 for c
In this case, -c6n2+12cn-8c+n2 = 0 leads to c = f(n) n2 (1 – 6c) + 12cn - 8c < 0Look for a combination of c and n0 that works in definition of big O
n2 (1 – 6c) will be the dominate term at large n Try c = 1 and see how large n must be
-5n2 + 12n - 8 < 0 true for all n>1c = n0 = 1 work in definition of big O
T(n) > cn3 - 6cn2 + 12cn -8c + n2 > cn3
Solve - 6cn2 + 12cn -8c + n2 > 0 for c
Look for a combination of c and n0 that works in definition of big W
c = 1/6 eliminates n2 term
2n – 4/3 > 0 true for all n>1c = 1/6 and n0 = 1 work in definition of big W
Master Theorem: (statement on p94)“cook-book” method to solve T(n) = aT(n/b) + f(n) when
a > 1, b > 1, and f(n) asymptotically positive
Ignore floors and ceiling do not effect validity (section 4.6.2 p103)
At large n, compare f(n) to(a)logn b
Case 1: (a)logn b
is polynomially larger than f(n) at large n
Conclusion: T(n) = Q( )(a)logn b
We can subtract e > 0 from logb(a) and(a)logn b
is still an upper bound on f(n)
There exist e > 0 such that f(n) = O( )(a)logn b
Case 2: (a)logn b
is asymptotically the same as f(n)
Conclusion:
(a)logn b )f(n) = Q(
(a)logn b lgn) = Q(f(n)lgn)T(n) = Q(
Case 3: (a)logn b
f(n) is polynomially larger than
There exist e > 0 such that f(n) = W( )(a)logn b
is still a lower bound on f(n)
f(n) is “regular” af(n/b) < cf(n) for some c < 1 and n > n0
Conclusion: T(n) = Q(f(n))
We can add e > 0 from logb(a) and (a)logn b
Cpt S 450 Spring 2015[All problems are from Cormen et al, 3rd Edition]
Homework Assignment 5: due 2/18/15Problems 4-1a, c and e on page 107 by master method
When is Master Theorem not applicable
1. T(n) does not have the correct form
2. f(n) falls between cases 1 and 2 (cannot find e > 0 required by case 1)
3. f(n) falls between cases 2 and 3 (cannot find e > 0 required by case 3) 4. Regularity condition fails in case 3
Quiz Friday 2-20-15Tree analysis
size of tree (istop)cost of leaves (number of leaves)cost of ith leveltotal cost of levelstotal costorder of growth
Proof of Master Theorem
A. Analyze master recurrence for n = bk (no floors or ceilings)
1. tree analysis → solution as a sum2. bound sum → 3 cases
B. Extend solution to any n > 0Apply techniques related to floors and ceilings
MT applies to T(n) = aT(n/b) + f(n) with a>1 and b>1
Case 1: cost of leaves dominates
Case 2: cost evenly distributed over levels and leaves
f(n)lg(n))())lg((a)log(nT(n) b n
Case 3: cost of root dominates: T(n) = Q(f(n))
T(n) = aT(n/b) + f(n)
bi bi
)(a)log(nT(n) b
When n is not a power of b recurrence becomes
T(n) = constant n=1T(n) = T(floor(n/2)) + T(ceiling(n/2)) + Q(n) n>1
Section 4.6.2 show that floors and ceiling do not affect the tree analysis
T(n) = aT(n/b) + f(n)
Quiz Friday, 2-28-14Master Theorem
Which case appliesRelationship between f(n) and nlogb(a) in asymptotic notationAcceptable values of e, cases 1 and 3Regularity of f(n), case 3
Iteration on recurrence T(n)=4T(n/2)+cn
T(n) = cn + 4T(n/2)T(n) = cn + 4(cn/2 + 4T(n/4))T(n) = cn +2cn +16(cn/4 + 4T(n/8))T(n) = cn +2cn + 4cn +64(cn/8+4T(n/16))T(n) = cn +2cn + 4cn + 8cn +256(cn/16 + 4T(n/32))Level = 0 1 2 3 …Cost of level k is 2kcnT(n)=Sk=0 to istop 2kcn=cn(2lg(n)+1-1)/(2-1)=cn(2nlg(2)-1)
Compare to solution by master’s theorem
Top Related