Chapter 4: Solution of recurrence relationships

Techniques:Substitution: Guess answer, prove by induction

Tree analysis: Good way to develop a guess

Master theorem: Recipe when T(n) = aT(n\b) + f(n)with a >1, b >1 and f(n) a specified function

Example: use substitution method to find run time for merge sort

When n is a power of 2

The recurrence for merge sort (worst case) is

T(n) = 2 n=2T(n) = 2T(n/2) +n n=2k k>1

Proved by induction on integers (HW1)T(n) = nlgn= 2k lg2k = k2k

Worst case merge sort for any n

T(n) = constant n=1T(n) = T(floor(n/2)) + T(ceiling(n/2)) + Q(n) n>1

Look for order of growth instead of analytic solution

We can ignore floor and ceiling and replace Q(n) by cn with c > 0

Recurrence becomes T(n) = 2T(n/2) + cn

Prove T(n) = Q(nlgn) by substitution method

Based on Theorem 3.1, T(n) = Q(nlgn) iff T(n) = O(nlgn) and T(n) = W(nlgn)

Start by proving T(n) = O(nlgn) is a solution of T(n) = 2T(n/2) + cn

if -dn + cn = 0; therefore d = c > 0 will work as the constant in the definition of big O

For a more general solution start with the string on inequalities

T(n) < dnlg(n)-dn+cn < dnlg(n),

which says

if T(n) < dnlg(n)-dn+cn

and dnlg(n)-dn+cn < dnlg(n)

then T(n) < dnlg(n),

which is what we need in order to show that T(n)=O(nlg(n)).

But T(n) < dnlg(n)-dn+cn < dnlg(n)

also put a condition on d by the inequality

dnlg(n)-dn+cn < dnlg(n)

Solving this inequality for d yields

d > c > 0,

which says that d = c is OK as the constant in the definition of big O but any d bigger than c is also OK

Finish by proving T(n) = W(nlgn) is a solution ofT(n) = 2T(n/2) + cnAssume T(n/2)= (W (n/2)lg(n/2))

if -dn + cn = 0; therefore d = c > 0 will work as the constant in the definition of big W

For a more general solution start with the string on inequalities

T(n) > dnlg(n)-dn+cn > dnlg(n),

which says

if T(n) > dnlg(n)-dn+cn

and dnlg(n)-dn+cn > dnlg(n)

then T(n) > dnlg(n),

which is what we need in order to show that T(n)=W(nlg(n)).

But T(n) > dnlg(n)-dn+cn > dnlg(n)

also put a condition on d by the inequality

dnlg(n)-dn+cn > dnlg(n)

Solving this inequality for d yields

0 < d < c,

which says that d = c is OK as the constant in the definition of big W but any d between 0 and c is also OK

Sometimes, even with correct guess, substitution does not work To prove Big O in these cases, try subtracting a lower-order term in your assumption

Example: T(n) = 8T(n/2) + Q(n2)Show T(n) = O(n3)

if d = c > 0

The string of inequalities T(n) < dn3 + cn2 < dn3 cannot be true because cn2 > 0

The substitution T(n/2) = O((n/2)3 will not work in this case

Try “subtract off a lower order term”assume T(n/2) < d[(n/2)3 – (n/2)2]

((n/2)3)

if d = c > 0

if we assume T(n/2) < d[(n/2)3 – (n/2)2] then we most show T(n) < d[n3-n2]

< d[n3-n2] implies

T(n) < d[n3-n2] and 0 < c < d (i.e. any d > c satisfies the definition of big O)

Substitution T(n/2) = W((n/2)3) works for proving T(n) = W(n3)is a solution of T(n) = 8T(n/2) + cn2 with c > 0

Assuming T(n/2) = W((n/2)3) means there exist d>0 such thatT(n/2) > d(n/2)3 which impliesT(n) > 8d(n/2)3 + cn2

The string of inequalities T(n) > dn3 + cn2 > dn3 can be true because cn2 > 0,which means that any d > 0 is OK as the constant in the definition of big ;W hence T(n) = W(n3)

Sometimes a change of variable helps

Example: T(n) = 2T(n1/2) + lg(n)

Change of variable in substitution method

Quiz Friday, 2-6-15Evaluate and bound sums

prove formulas by induction on integersconvert one sum into anotherbound by inductionbound every termbound by integration

CptS 450 Spring 2015[All problems are from Cormen et al, 3rd Edition] Homework Assignment 4: due 2/11/151. ex 4.4-7 on page 93 (tree analysis and substitution)

2. problem 4-3f on page 108 by tree analysis and substitution

Solution of recurrence by tree analysis

Tree analysisAt each node write overhead per recurrence

usually a function of level indexCost of leaves = number of leavesCalculate istopSum level cost for i = 0 to istop -1Add cost of leaves

If tree analysis is ambiguous, verify by substitution

Example: T(n) = 3T(|_n/4_|) + Q(n2) ~ 3T(n/4) + cn2

T(n)=3T(n/4)+cn2 cost of level i

istop = ? How many leaves?

As indicated in previous slide, levels are indexed from zero

ISTOP is the index of leavesAt ISTOP divide and conquer has bottomed out with subgroups of size n0

Could evaluate cost of level using finite geometric sum

Bounding by infinite series easier

series

Test guess from tree analysis by substitution

Test guess from tree analysis by substitution

Guess is upper bound onlyDid not evaluate sum over levels

= 1

solving right-most inequality for d yields d > (16/13)c > 0 for all n > 1there exists d > 0 such that 0 < T(n) < dn2 all n > 1hence, T(n) = O(n2)

< dn2

Binary tree with complicated level cost

T(n) = 2T(n/2 +17) +n

Expect T(n) = nlgn Why?

Note: sum cannot be bounded by geometric series. Why?

HW 4 due date changed

Homework Assignment 4: due 2/13/151. ex 4.4-7 on page 93 (tree analysis and substitution)

2. problem 4-3f on page 108 by tree analysis and substitution

Quiz Friday 2/13Substitution method

set up (remove floors, ceiling, asymptotic notation)assumptionimplementation of assumptionconstants and n0

Imbalanced tree analysis: T(n) = T(n/3) + T(2n/3) + cn

Imbalanced tree analysis: T(n) = T(n/3) + T(2n/3) + cn

Imbalanced tree analysis continued

Test guess by substitution

< 0 for d

= all n > 0

< dnlgn

solve

d > c/(lg3-2/3) > 0

Skinny Trees

Skinny tree

leaf = no2

no2

no2

no2 + Q(n3)

See text p1147

substitution

= 0

< cn3

T(n) < cn3 - 6cn2 + 12cn -8c + n2 < cn3

Solve - 6cn2 + 12cn -8c + n2 < 0 for c

In this case, -c6n2+12cn-8c+n2 = 0 leads to c = f(n) n2 (1 – 6c) + 12cn - 8c < 0Look for a combination of c and n0 that works in definition of big O

n2 (1 – 6c) will be the dominate term at large n Try c = 1 and see how large n must be

-5n2 + 12n - 8 < 0 true for all n>1c = n0 = 1 work in definition of big O

T(n) > cn3 - 6cn2 + 12cn -8c + n2 > cn3

Solve - 6cn2 + 12cn -8c + n2 > 0 for c

Look for a combination of c and n0 that works in definition of big W

c = 1/6 eliminates n2 term

2n – 4/3 > 0 true for all n>1c = 1/6 and n0 = 1 work in definition of big W

Master Theorem: (statement on p94)“cook-book” method to solve T(n) = aT(n/b) + f(n) when

a > 1, b > 1, and f(n) asymptotically positive

Ignore floors and ceiling do not effect validity (section 4.6.2 p103)

At large n, compare f(n) to(a)logn b

Case 1: (a)logn b

is polynomially larger than f(n) at large n

Conclusion: T(n) = Q( )(a)logn b

We can subtract e > 0 from logb(a) and(a)logn b

is still an upper bound on f(n)

There exist e > 0 such that f(n) = O( )(a)logn b

Case 2: (a)logn b

is asymptotically the same as f(n)

Conclusion:

(a)logn b )f(n) = Q(

(a)logn b lgn) = Q(f(n)lgn)T(n) = Q(

Case 3: (a)logn b

f(n) is polynomially larger than

There exist e > 0 such that f(n) = W( )(a)logn b

is still a lower bound on f(n)

f(n) is “regular” af(n/b) < cf(n) for some c < 1 and n > n0

Conclusion: T(n) = Q(f(n))

We can add e > 0 from logb(a) and (a)logn b

Cpt S 450 Spring 2015[All problems are from Cormen et al, 3rd Edition]

Homework Assignment 5: due 2/18/15Problems 4-1a, c and e on page 107 by master method

When is Master Theorem not applicable

1. T(n) does not have the correct form

2. f(n) falls between cases 1 and 2 (cannot find e > 0 required by case 1)

3. f(n) falls between cases 2 and 3 (cannot find e > 0 required by case 3) 4. Regularity condition fails in case 3

Prove that, in case 3, polynomial f(n) is regular

ε (a)logn b Given f(n) = nk and f(n) = W( )

Prove that, in case 3, polynomial f(n) is regular

Quiz Friday 2-20-15Tree analysis

size of tree (istop)cost of leaves (number of leaves)cost of ith leveltotal cost of levelstotal costorder of growth

Proof of Master Theorem

A. Analyze master recurrence for n = bk (no floors or ceilings)

1. tree analysis → solution as a sum2. bound sum → 3 cases

B. Extend solution to any n > 0Apply techniques related to floors and ceilings

MT applies to T(n) = aT(n/b) + f(n) with a>1 and b>1

Tree analysis master theorem T(n) = aT(n/b) + f(n)

bi bi

Case 1: cost of leaves dominates

Case 2: cost evenly distributed over levels and leaves

f(n)lg(n))())lg((a)log(nT(n) b n

Case 3: cost of root dominates: T(n) = Q(f(n))

T(n) = aT(n/b) + f(n)

bi bi

)(a)log(nT(n) b

When n is not a power of b recurrence becomes

T(n) = constant n=1T(n) = T(floor(n/2)) + T(ceiling(n/2)) + Q(n) n>1

Section 4.6.2 show that floors and ceiling do not affect the tree analysis

T(n) = aT(n/b) + f(n)

Quiz Friday, 2-28-14Master Theorem

Which case appliesRelationship between f(n) and nlogb(a) in asymptotic notationAcceptable values of e, cases 1 and 3Regularity of f(n), case 3

Iteration on recurrence T(n)=4T(n/2)+cn

T(n) = cn + 4T(n/2)T(n) = cn + 4(cn/2 + 4T(n/4))T(n) = cn +2cn +16(cn/4 + 4T(n/8))T(n) = cn +2cn + 4cn +64(cn/8+4T(n/16))T(n) = cn +2cn + 4cn + 8cn +256(cn/16 + 4T(n/32))Level = 0 1 2 3 …Cost of level k is 2kcnT(n)=Sk=0 to istop 2kcn=cn(2lg(n)+1-1)/(2-1)=cn(2nlg(2)-1)

Compare to solution by master’s theorem

More iteration examples

T(n)=T(n-2)+n2

T(n)=T(n/2)+T(n/4)+T(n/8)