Recurrences 2008. 1. 11 : 1 Chapter 3. Growth of function Chapter 4. Recurrences.

1

Recurrences

한양대학교 정보보호 및 알고리즘 연구실

2008. 1. 11

이재준

담당교수님 : 박희진 교수님

Chapter 3. Growth of functionChapter 4. Recurrences

Contents of Table

1. Review2. Asymptotic notation3. The substitution method4. The recursion-tree method5. The master theorem method6. Proof of the master theorem

2

1. Review

3

Insertion Sort Merge Sort

Asymptotic Complexity

Insertion Sort Merge Sort

Best, Worst, Average case draw and understandrecursion tree

Address logically to audience

4

• Theta[θ]-notation

- [Theta] : f(n) = θ(g(n))

- for all n, n ≥ n0 ,

If there exist positive constant c1 , c2

and n0 such that

c1 g(n) ≤ f(n) ≤ c2 g(n) then,

f(n) = θ(g(n)).

- if g(n) is both an upper and lower bound on f(n) then,

f(n) = g(n).

2. Asymptotic notation

5

• Big Oh[O]-notation

- [Big Oh] : f(n) = O(g(n))


If there exist positive constant c and n0 such that

f(n) ≤ c g(n) then,

f(n) = O(g(n)).


6

• Omega [Ω]-notation

- [Omega] : f(n) = Ω (g(n))


If there exist positive constant c and n0

such that

c g(n) ≥ f(n) then,

f(n) = Ω(g(n)).


Recurrences

7

• Recurrence - Definition Equation or inequality that describes a function in terms of

its value on smaller inputs.

- Recurrence Solution Method ☞ substitution method ☞ recursion-tree method

☞ master method

)()2/(2

)1()(

nnTnT

if n=1

if n>1

Recurrences

• Technicalities - Omit Technical Details ☞ assumption of integer ☞ floors

☞ ceilings ☞ boundary condition

- Example (Merge Sort)

)()2/(2

)1()(

nnTnT

if n=1

if n>1

if n=1

if n>1

)()2/()2/(

)1()(

nnTnTnT

)()2/(2)( nnTnT Assumption of inte-ger

Boundary condition

Floors and ceil-ings

3. The substitution method

9

• Substitution Method - Two steps

- It can be used to establish either upper or lower bounds on a recurrence.

1. Guess the form of the solution 2. Use mathematical induction to find the constant

and show that the solution works.


10

• Example

• Step for substitution method

1. guess that the solution is 2. prove that ( c >0 )

)lg()( nnnT

ncnnT lg)(

nnTnT )2/(2)(


11

- Assume that this bound holds for ⌊n/2 , ⌋ that is, T (⌊n/2 ) ≤ ⌋ c ⌊n/2 lg(⌋ ⌊n/2 ).⌋

T(n) ≤ 2(c ⌊n/2 lg(⌋ ⌊n/2 )) + ⌋ n

≤ cn lg(n/2) + n ( because, ⌊n/2 < ⌋ n/2 )

= cn lg n - cn lg 2 + n = cn lg n - cn + n ≤ cn lg n (as long as c ≥ 1)


• Find the constant n0

- we can take advantage of asymptotic notation,we can replace base case

T(1) ≤ cn lg n

T(1) = 1 but, c1 lg1 = 0

- So, we only have to prove T (n) = cn lg n for n ≥ n0 for n (n0 =2)

12

Replace T(1) by T(2) and T(3) as the base cases by letting n0 =2.


• Find the constant c - choosing c large enough

T (2) = 4 and T (3) = 5.

T (2) = 4 ≤ c 2 lg 2

T (3) = 5 ≤ c 3 lg 3.

13

Any choice of c ≥ 2 suffices for base case of n ≥ n0 , n0 =2

4. The recursion-tree method

• Recursion-tree - Definition Each node represents the cost of a single subprogram

somewhere in the set of recursive function invocation - The reason why we use this method ☞ A recursion-tree is best used to generate a good guess ☞ Use directly to prove the master theorem

- Total Cost We sum the cost each level of the tree to obtain a set of per-

level costs, and then we sum all the per-level costs to de-termine the total cost of all levels of the recursion

14


• Example

• Draw recursion tree

15

cn2

)4

(n

T

T(n)

)4

(n

T )4

(n

T

)16

(n

T )16

(n

T )16

(n

T

2)4

(n

c

)16

(n

T )16

(n

T )16

(n

T

2)4

(n

c

)16

(n

T )16

(n

T )16

(n

T

2)4

(n

c

cn2

T (n) = 3T (⌊n/4 ) + Θ(⌋ n2)

< 3T (n/4) + cn2


• Recursion-tree

16

cn2

c(n/4)2 c(n/4)2 c(n/4)2

2)16

(n

c 2)16

(n

c 2)16

(n

c 2)16

(n

c 2)16

(n

c 2)16

(n

c 2)16

(n

c 2)16

(n

c 2)16

(n

c

T(1) T(1)T(1) T(1) T(1) T(1)T(1) T(1) T(1) T(1)T(1) T(1)…

Depth

Sub problem size

0 n

1 n/4

2 n/16

.

.

.

.

.

.

.

i n/4i = 1

3log4n

log4n


17

• Total Cost - When depth i then subprogram size is n/4i = 1 so, i = log4 n

and it means tree’s height and tree has log4 n + 1 levels.

- The number of nodes at depth i is 3i and finally - Total Cost is

)(1)16/3(

1)16/3(

)()16

3(

)()16

3(...)

16

3(

16

3)(

3log2log

3log21log

0

3log21log2222

4

4

4

4

44

ncn

ncn

ncncncncnnT

n

in

i

n

Sum of cost for root to i-1 depth

Sum of cost for i depth

3loglog 443 nn


18

• Total Cost is..

)(

)(13

16

)()16/3(1

1

)()16

3(

)()16

3()(

2

3log2

3log2

3log2

0

3log21log

0

4

4

4

4

4

nO

ncn

ncn

ncn

ncnnT

i

i

in

i

xx

k

i

1

1

0


19

• Prove T (n) = O(n2) by the substitution method

- Problem

- Step for substitution method

T (n) = 3T (⌊n/4 ) + Θ(⌋ n2)

1. guess that the solution is T (n) = O(n2) 2. prove that T (n) ≤ dn2 (for some d > 0 and for the same c > 0)


20

• Prove.. Find constant d

T(n) ≤ 3T(⌊n/4 ) + ⌋ cn2

≤ 3d⌊n/4⌋2 + cn2

≤ 3d(n/4)2 + cn2 = 3/16 dn2 + cn2

≤ dn2 ( where the last step holds as long as d ≥ (16/13)c )

5. The master theorem method

• Master theorem - Definition method for solving recurrences of the form where a ≥ 1 and

b > 1 are constant and f (n) is an asymptotically positive function

21

)()/()( nfbnaTnT

The cost of dividing the program and com-bining

Subprograms are solved recur-sively


• Master theorem

22

)()/(

)1()(

nfbnaTnT

if n=1

if n>1

)()/()( nfbnaTnT

)()/()()( log1log

0

ajjn

j

b

b

nbnfanT

By Assumption of integer and Boundary con-dition

By Draw recursion tree


23

• Three case of master theorem The three case of master theorem correspond to cases in

which the total cost of the tree.

- Case 1 : dominated by the costs in the leaves. <

- Case 2 : distributed across the levels of the tree. =

- Case 3 : dominated by the cost of the root. >

)(nf abnlog

)(nf abnlog

)(nf abnlog

)()( log abnOnf

)()( log abnnf

)()( log abnnf

)()( log abnnT

)lg()( log nnnT ab

))(()( nfnT


• Master theorem

24

Let a ≥ 1 and b > 1 are constants, let f (n) be a function, and let

T (n) be defined on the nonnegative integers by the recurrence

where we interpret n/b to mean either or . Then T (n) can be bounded asymptotically as follows.

1. If for some constant ε > 0, then . 2. If , then .3. If for some constant ε > 0, and if af (n/b) ≤ cf (n) for some constant c < 1 and all sufficiently large n, then .

)()/()( nfbnaTnT

bn / bn /

)()( log abnOnf)()( log abnnf

)()( log abnnf

))(()( nfnT

)()( log abnnT )lg()( log nnnT ab


25

• Exception in case of master theo-rem

- Exception A : Gap between cases 1 and 2 when < but not polynomially smaller.

- Exception B : Gap between cases 2 and 3 when >

but not polynomially larger.

)(nf abnlog

)(nf abnlog


26

• Exception in case of master theorem • a=2, b=2, f (n) = n lg n, and it might be case 3?• Asymptotically larger f (n) = n lg n

> (asymptotically larger)

• But not Polynomially larger

The ratio is asymptotically less than for any positive constant k (polynomially smaller)

• Recurrence falls into gap between cases 2 and 3.

nnnTnT lg)2/(2)( nnn ab 2loglog 2

nnn ab 2loglog 2

nnnnnnf ab lg/)lg(/)( log kn

6. The proof of master theorem

27

• Case 1 : dominated by the costs in the leaves.

we have for some constant ,then so, ,which implies that ..

)/()()(1log

0

jjn

j

bnfangb

)()( log abnOnf 0 )()( log abnng

abnjj bnObnf

log

)/()/(

)( log

)()()(1log

0

ab

bn

jj

n

j b

naOng


28

• Case 1 : dominated by the costs in the leaves.

)( log

)()()(1log

0

ab

b

jj

n

j b

naOng

Constant value

)( log abnO


29

• Case 2 : distributed across the levels of the tree.

)( log1log

0

)()()( aj

jn

j

b

b

b

nang

)()( log abnnf

)( lglog nn ab


30

• Case 3 : dominated by the cost of the root. We assume that af (n/b) ≤ cf (n) for some constant and c < 1, all n

≥ b,

)()/(

)()(c/a )/(

)(/a)( )/(

nfcbnfa

nfbnf

nfcbnf

jjj

jj

( Iterating j times )

Thank you

Q & A

31

Recurrences 2008. 1. 11 : 1 Chapter 3. Growth of function Chapter 4. Recurrences.

Documents

Transcript of Recurrences 2008. 1. 11 : 1 Chapter 3. Growth of function Chapter 4. Recurrences.