CS 2133:Data Structures Merge Sort Solving Recurrences The Master Theorem.

CS 2133:Data Structures

Merge Sort

Solving Recurrences

The Master Theorem

Merge Sort

MergeSort(A, left, right) {

if (left < right) {

mid = floor((left + right) / 2);

MergeSort(A, left, mid);

MergeSort(A, mid+1, right);

Merge(A, left, mid, right);

}

}

// Merge() takes two sorted subarrays of A and

// merges them into a single sorted subarray of A

// (how long should this take?)

Merge Sort: Example

Show MergeSort() running on the array

A = {10, 5, 7, 6, 1, 4, 8, 3, 2, 9};

Analysis of Merge Sort

Statement Effort

So T(n) = (1) when n = 1, and

2T(n/2) + (n) when n > 1 So what (more succinctly) is T(n)?

MergeSort(A, left, right) { T(n) if (left < right) { (1) mid = floor((left + right) / 2); (1) MergeSort(A, left, mid); T(n/2) MergeSort(A, mid+1, right); T(n/2) Merge(A, left, mid, right); (n) }}

Recurrences

The expression:

is a recurrence. Recurrence: an equation that describes a function

in terms of its value on smaller functions

1

22

1

)(nc

nT

nc

nT

Recurrence Examples

0

0

)1(

0)(

n

n

nscns

0)1(

00)(

nnsn

nns

1

22

1

)(nc

nT

nc

nT

1

1

)(

ncnb

naT

nc

nT

Solving Recurrences

Substitution method Iteration method Master method

Solving Recurrences

The substitution method A.k.a. the “making a good guess method” Guess the form of the answer, then use induction

to find the constants and show that solution works This and several other methods are usually done in

discrete math so we will skip them at this time.

Solving Recurrences

Another option is what the book calls the “iteration method” Expand the recurrence Work some algebra to express as a summation Evaluate the summation

We will show several examples

s(n) =

c + s(n-1)

c + c + s(n-2)

2c + s(n-2)

2c + c + s(n-3)

3c + s(n-3)

…

kc + s(n-k) = ck + s(n-k)

0)1(

00)(

nnsc

nns

So far for n >= k we have s(n) = ck + s(n-k)

What if k = n? s(n) = cn + s(0) = cn

0)1(

00)(

nnsc

nns

So far for n >= k we have s(n) = ck + s(n-k)

What if k = n? s(n) = cn + s(0) = cn

So

Thus in general s(n) = cn

0)1(

00)(

nnsc

nns

0)1(

00)(

nnsc

nns

s(n)

= n + s(n-1)

= n + n-1 + s(n-2)

= n + n-1 + n-2 + s(n-3)

= n + n-1 + n-2 + n-3 + s(n-4)

= …

= n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)

0)1(

00)(

nnsn

nns

s(n)

= n + s(n-1)

= n + n-1 + s(n-2)

= n + n-1 + n-2 + s(n-3)

= n + n-1 + n-2 + n-3 + s(n-4)

= …

= n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)

=

0)1(

00)(

nnsn

nns

)(1

knsin

kni

So far for n >= k we have

0)1(

00)(

nnsn

nns

)(1

knsin

kni


What if k = n?

0)1(

00)(

nnsn

nns

)(1

knsin

kni


What if k = n?

0)1(

00)(

nnsn

nns

)(1

knsin

kni

2

10)0(

11

nnisi

n

i

n

i


What if k = n?

Thus in general

0)1(

00)(

nnsn

nns

)(1

knsin

kni

2

10)0(

11

nnisi

n

i

n

i

2

1)(

nnns

T(n) =

2T(n/2) + c

2(2T(n/2/2) + c) + c

22T(n/22) + 2c + c

22(2T(n/22/2) + c) + 3c

23T(n/23) + 4c + 3c

23T(n/23) + 7c

23(2T(n/23/2) + c) + 7c

24T(n/24) + 15c

…

2kT(n/2k) + (2k - 1)c

1

22

1)( nc

nT

ncnT

So far for n > 2k we have T(n) = 2kT(n/2k) + (2k - 1)c

What if k = lg n? T(n) = 2lg n T(n/2lg n) + (2lg n - 1)c

= n T(n/n) + (n - 1)c

= n T(1) + (n-1)c

= nc + (n-1)c = (2n - 1)c

1

22

1)( nc

nT

ncnT

T(n) =

aT(n/b) + cn

a(aT(n/b/b) + cn/b) + cn

a2T(n/b2) + cna/b + cn

a2T(n/b2) + cn(a/b + 1)

a2(aT(n/b2/b) + cn/b2) + cn(a/b + 1)

a3T(n/b3) + cn(a2/b2) + cn(a/b + 1)

a3T(n/b3) + cn(a2/b2 + a/b + 1)

…

akT(n/bk) + cn(ak-1/bk-1 + ak-2/bk-2 + … + a2/b2 + a/b + 1)

1

1)( ncn

b

naT

ncnT

So we have T(n) = akT(n/bk) + cn(ak-1/bk-1 + ... + a2/b2 + a/b + 1)

For k = logb n n = bk

T(n) = akT(1) + cn(ak-1/bk-1 + ... + a2/b2 + a/b + 1)

= akc + cn(ak-1/bk-1 + ... + a2/b2 + a/b + 1)

= cak + cn(ak-1/bk-1 + ... + a2/b2 + a/b + 1)

= cnak /bk + cn(ak-1/bk-1 + ... + a2/b2 + a/b + 1)

= cn(ak/bk + ... + a2/b2 + a/b + 1)

1

1)( ncn

b

naT

ncnT

So with k = logb n T(n) = cn(ak/bk + ... + a2/b2 + a/b + 1)

What if a = b? T(n) = cn(k + 1)

= cn(logb n + 1)

= (n log n)

1

1)( ncn

b

naT

ncnT


What if a < b?

1

1)( ncn

b

naT

ncnT


What if a < b? Recall that (xk + xk-1 + … + x + 1) = (xk+1 -1)/(x-1)

1

1)( ncn

b

naT

ncnT


What if a < b? Recall that (xk + xk-1 + … + x + 1) = (xk+1 -1)/(x-1) So:

1

1)( ncn

b

naT

ncnT

baba

ba

ba

ba

b

a

b

a

b

a kk

k

k

k

k

1

1

1

1

1

11

11

1

1


What if a < b? Recall that (xk + xk-1 + … + x + 1) = (xk+1 -1)/(x-1) So:

T(n) = cn ·(1) = (n)

1

1)( ncn

b

naT

ncnT

baba

ba

ba

ba

b

a

b

a

b

a kk

k

k

k

k

1

1

1

1

1

11

11

1

1


What if a > b?

1

1)( ncn

b

naT

ncnT


What if a > b?

1

1)( ncn

b

naT

ncnT

k

k

k

k

k

k

baba

ba

b

a

b

a

b

a

1

11

1

1

1


What if a > b?

T(n) = cn · (ak / bk)

1

1)( ncn

b

naT

ncnT

k

k

k

k

k

k

baba

ba

b

a

b

a

b

a

1

11

1

1

1


What if a > b?

T(n) = cn · (ak / bk)

= cn · (alog n / blog n) = cn · (alog n / n)

1

1)( ncn

b

naT

ncnT

k

k

k

k

k

k

baba

ba

b

a

b

a

b

a

1

11

1

1

1


What if a > b?

T(n) = cn · (ak / bk)


recall logarithm fact: alog n = nlog a

1

1)( ncn

b

naT

ncnT

k

k

k

k

k

k

baba

ba

b

a

b

a

b

a

1

11

1

1

1


What if a > b?

T(n) = cn · (ak / bk)



= cn · (nlog a / n) = (cn · nlog a / n)

1

1)( ncn

b

naT

ncnT

k

k

k

k

k

k

baba

ba

b

a

b

a

b

a

1

11

1

1

1


What if a > b?

T(n) = cn · (ak / bk)



= cn · (nlog a / n) = (cn · nlog a / n)

= (nlog a )

1

1)( ncn

b

naT

ncnT

k

k

k

k

k

k

baba

ba

b

a

b

a

b

a

1

11

1

1

1

So…

1

1)( ncn

b

naT

ncnT

ca

cb

c

ban

bann

ban

nTblog

log)(

A Simple form of the Master Method

Using The Master Method

T(n) = 9T(n/3) + n a=9, b=3, c = 1 Now 9 > 31

Thus the solution is T(n) = (n2)T(n) = 2T(n/2) + n

a=2, b=2, c = 1

Now 2= 21

Thus the solution is T(n) = (n log n)

Homework

Go to my web page and download homework 2.this is a collection of recurrence relations that youwill apply the master method and the iterationmethod to.

CS 2133:Data Structures Merge Sort Solving Recurrences The Master Theorem.

Documents

Transcript of CS 2133:Data Structures Merge Sort Solving Recurrences The Master Theorem.