1

Chapter 4 Divide-and-Conquer

2

About this lecture

bull Recall the divide-and-conquer paradigm which we used for merge sortndash Divide the problem into a number of sub-problems

that are smaller instances of the same problemndash Conquer the sub-problems by solving them

recursivelybull Base case If the sub-problems are small enough just solve

them by brute forcendash Combine the sub-problem solutions to give a solution

to the original problembull We look at two more algorithms based on divide-

and-conquer

3

About this lecture

bull Analyzing divide-and-conquer algorithmsbull Introduce some ways of solving recurrencesndash Substitution Method (If we know the answer)ndash Recursion Tree Method (Very useful )ndash Master Theorem (Save our effort)

4

Maximum-subarray problem

bull Input an array A[1n] of n numbersndash Assume that some of the numbers are negative

because this problem is trivial when all numbers are nonnegative

bull Output a nonempty subarray A[ij] having the largest sum S[i j] = ai + ai+1 + + aj

13 -3 -25 20 -3 -16 -23 18 20 -7 12 -5 -22 15 -4 7

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

A

maximum subarray

5

A brute-force solutionbull Examine all possible S[i j]bull Two implementationsndash compute each S[i j] in O(n) time O(n3) timendash compute each S[i j+1] from S[i j] in O(1) timendash (S[i i] = A[i] and S[i j+1] = S[i j] + A[j+1])ndash O(n2) time Ex i 1 2 3 4 5 6 A[i] 13 -3 -25 20 -3 -16

S[1 j] = 13 10 -15 5 2 -14 S[2 j] = -3 -28 -8 -11 -27

2

n

6

A divide-and-conquer solutionbull Possible locations of a maximum subarray A[ij] of

A[lowhigh] where mid = (low+high)2ndash entirely in A[lowmid] (low i j mid)ndash entirely in A[mid+1high] (mid lt i j high)ndash crossing the midpoint (low i mid lt j high)

low mid high

mid +1

entirely in A[lowmid] entirely in A[mid+1high]

crossing the midpoint

Possible locations of subarrays of A[lowhigh]

7

low mid high

mid +1

A[imid]

j

A[mid+1j]

i

A[ij] comprises two subarrays A[imid] and A[mid+1j]

8

FIND-MAX-CROSSING-SUBARRAY (A low mid high)

left-sum = - 1048576 Find a maximum subarray of the form A[imid] sum = 0for i = mid downto low sum = sum + A[i ] if sum gt left-sum left-sum = sum max-left = i right-sum = - 1048576 Find a maximum subarray of the form A[mid + 1 j ]sum =0for j = mid +1 to high sum = sum + A[j] if sum gt right-sum right-sum = sum max-right = j Return the indices and the sum of the two subarraysReturn (max-left max-right left-sum + right-sum)

9

1 2 3 4 5 6 7 8 9 10

13 -3 -25 20 -3 -16 -23 18 20 -7

Example

A

S[5 5] = -3S[4 5] = 17 (max-left = 4)S[3 5] = -8S[2 5] = -11S[1 5] = 2

mid =5

1 2 3 4 5 6 7 8 9 10

13 -3 -25 20 -3 -16 -23 18 20 -7A

S[6 6] = -16S[6 7] = -39S[6 8] = -21S[6 9] = (max-right = 9) -1S[610] = -8

mid =5

maximum subarray crossing mid is S[49] = 16

10

if high == lowReturn (low high A[low]) base case only one element

else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)

FIND-MAXIMUM-SUBARRAY (A low high)

Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)

2highlow

11

Analyzing time complexity

bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY

T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)

12

Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB

1

kj

n

kikij bac

An O(n3) time naive algorithm

SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj

return C

13

Divide-and-Conquer Algorithm

bull Assume that n is an exact power of 2

2221

1211

2221

1211

2221

1211 CC

CCC

BB

BBB

AA

AAA

2221

1211

2221

1211

2221

1211

BB

BB

AA

AA

CC

CC

(41)

14

Divide-and-Conquer Algorithm

2112111111 BABAC

2122112121 BABAC

2212121112 BABAC

2222122122 BABAC

A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)

= (n3)Computing A+B O(n2)

15

Strassenrsquos method

222131211222121 AASAASBBS

221162211511214 BBSAASBBS

211192221822127 AASBBSAAS

121110 BBS (42)

16

Strassenrsquos method

1111 SAP

2222 BSP

1133 BSP

4224 SAP

655 SSP

876 SSP

1097 SSP

624511 PPPPC

2112 PPC

4321 PPC

731522 PPPPC

(43)

(44)

17

Strassenrsquos divide-and-conquer algorithm

bull Step 1 Divide each of A B and C into four sub-matrices as in (41)

bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)

bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)

bull Step 4 Compute according to (44)

22211211 CCCC

18

Time complexity

T(n) = 7T(n2) + (n2)

= (nlg 7 ) (why)

= (n281)

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

2

About this lecture

bull Recall the divide-and-conquer paradigm which we used for merge sortndash Divide the problem into a number of sub-problems

that are smaller instances of the same problemndash Conquer the sub-problems by solving them

recursivelybull Base case If the sub-problems are small enough just solve

them by brute forcendash Combine the sub-problem solutions to give a solution

to the original problembull We look at two more algorithms based on divide-

and-conquer

3

About this lecture

bull Analyzing divide-and-conquer algorithmsbull Introduce some ways of solving recurrencesndash Substitution Method (If we know the answer)ndash Recursion Tree Method (Very useful )ndash Master Theorem (Save our effort)

4

Maximum-subarray problem

bull Input an array A[1n] of n numbersndash Assume that some of the numbers are negative

because this problem is trivial when all numbers are nonnegative

bull Output a nonempty subarray A[ij] having the largest sum S[i j] = ai + ai+1 + + aj

13 -3 -25 20 -3 -16 -23 18 20 -7 12 -5 -22 15 -4 7

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

A

maximum subarray

5

A brute-force solutionbull Examine all possible S[i j]bull Two implementationsndash compute each S[i j] in O(n) time O(n3) timendash compute each S[i j+1] from S[i j] in O(1) timendash (S[i i] = A[i] and S[i j+1] = S[i j] + A[j+1])ndash O(n2) time Ex i 1 2 3 4 5 6 A[i] 13 -3 -25 20 -3 -16

S[1 j] = 13 10 -15 5 2 -14 S[2 j] = -3 -28 -8 -11 -27

2

n

6

A divide-and-conquer solutionbull Possible locations of a maximum subarray A[ij] of

A[lowhigh] where mid = (low+high)2ndash entirely in A[lowmid] (low i j mid)ndash entirely in A[mid+1high] (mid lt i j high)ndash crossing the midpoint (low i mid lt j high)

low mid high

mid +1

entirely in A[lowmid] entirely in A[mid+1high]

crossing the midpoint

Possible locations of subarrays of A[lowhigh]

7

low mid high

mid +1

A[imid]

j

A[mid+1j]

i

A[ij] comprises two subarrays A[imid] and A[mid+1j]

8

FIND-MAX-CROSSING-SUBARRAY (A low mid high)

left-sum = - 1048576 Find a maximum subarray of the form A[imid] sum = 0for i = mid downto low sum = sum + A[i ] if sum gt left-sum left-sum = sum max-left = i right-sum = - 1048576 Find a maximum subarray of the form A[mid + 1 j ]sum =0for j = mid +1 to high sum = sum + A[j] if sum gt right-sum right-sum = sum max-right = j Return the indices and the sum of the two subarraysReturn (max-left max-right left-sum + right-sum)

9

1 2 3 4 5 6 7 8 9 10

13 -3 -25 20 -3 -16 -23 18 20 -7

Example

A

S[5 5] = -3S[4 5] = 17 (max-left = 4)S[3 5] = -8S[2 5] = -11S[1 5] = 2

mid =5

1 2 3 4 5 6 7 8 9 10

13 -3 -25 20 -3 -16 -23 18 20 -7A

S[6 6] = -16S[6 7] = -39S[6 8] = -21S[6 9] = (max-right = 9) -1S[610] = -8

mid =5

maximum subarray crossing mid is S[49] = 16

10

if high == lowReturn (low high A[low]) base case only one element

else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)

FIND-MAXIMUM-SUBARRAY (A low high)

Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)

2highlow

11

Analyzing time complexity

bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY

T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)

12

Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB

1

kj

n

kikij bac

An O(n3) time naive algorithm

SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj

return C

13

Divide-and-Conquer Algorithm

bull Assume that n is an exact power of 2

2221

1211

2221

1211

2221

1211 CC

CCC

BB

BBB

AA

AAA

2221

1211

2221

1211

2221

1211

BB

BB

AA

AA

CC

CC

(41)

14

Divide-and-Conquer Algorithm

2112111111 BABAC

2122112121 BABAC

2212121112 BABAC

2222122122 BABAC

A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)

= (n3)Computing A+B O(n2)

15

Strassenrsquos method

222131211222121 AASAASBBS

221162211511214 BBSAASBBS

211192221822127 AASBBSAAS

121110 BBS (42)

16

Strassenrsquos method

1111 SAP

2222 BSP

1133 BSP

4224 SAP

655 SSP

876 SSP

1097 SSP

624511 PPPPC

2112 PPC

4321 PPC

731522 PPPPC

(43)

(44)

17

Strassenrsquos divide-and-conquer algorithm

bull Step 1 Divide each of A B and C into four sub-matrices as in (41)

bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)

bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)

bull Step 4 Compute according to (44)

22211211 CCCC

18

Time complexity

T(n) = 7T(n2) + (n2)

= (nlg 7 ) (why)

= (n281)

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

3

About this lecture

bull Analyzing divide-and-conquer algorithmsbull Introduce some ways of solving recurrencesndash Substitution Method (If we know the answer)ndash Recursion Tree Method (Very useful )ndash Master Theorem (Save our effort)

4

Maximum-subarray problem

bull Input an array A[1n] of n numbersndash Assume that some of the numbers are negative

because this problem is trivial when all numbers are nonnegative

bull Output a nonempty subarray A[ij] having the largest sum S[i j] = ai + ai+1 + + aj

13 -3 -25 20 -3 -16 -23 18 20 -7 12 -5 -22 15 -4 7

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

A

maximum subarray

5

A brute-force solutionbull Examine all possible S[i j]bull Two implementationsndash compute each S[i j] in O(n) time O(n3) timendash compute each S[i j+1] from S[i j] in O(1) timendash (S[i i] = A[i] and S[i j+1] = S[i j] + A[j+1])ndash O(n2) time Ex i 1 2 3 4 5 6 A[i] 13 -3 -25 20 -3 -16

S[1 j] = 13 10 -15 5 2 -14 S[2 j] = -3 -28 -8 -11 -27

2

n

6

A divide-and-conquer solutionbull Possible locations of a maximum subarray A[ij] of

A[lowhigh] where mid = (low+high)2ndash entirely in A[lowmid] (low i j mid)ndash entirely in A[mid+1high] (mid lt i j high)ndash crossing the midpoint (low i mid lt j high)

low mid high

mid +1

entirely in A[lowmid] entirely in A[mid+1high]

crossing the midpoint

Possible locations of subarrays of A[lowhigh]

7

low mid high

mid +1

A[imid]

j

A[mid+1j]

i

A[ij] comprises two subarrays A[imid] and A[mid+1j]

8

FIND-MAX-CROSSING-SUBARRAY (A low mid high)

left-sum = - 1048576 Find a maximum subarray of the form A[imid] sum = 0for i = mid downto low sum = sum + A[i ] if sum gt left-sum left-sum = sum max-left = i right-sum = - 1048576 Find a maximum subarray of the form A[mid + 1 j ]sum =0for j = mid +1 to high sum = sum + A[j] if sum gt right-sum right-sum = sum max-right = j Return the indices and the sum of the two subarraysReturn (max-left max-right left-sum + right-sum)

9

1 2 3 4 5 6 7 8 9 10

13 -3 -25 20 -3 -16 -23 18 20 -7

Example

A

S[5 5] = -3S[4 5] = 17 (max-left = 4)S[3 5] = -8S[2 5] = -11S[1 5] = 2

mid =5

1 2 3 4 5 6 7 8 9 10

13 -3 -25 20 -3 -16 -23 18 20 -7A

S[6 6] = -16S[6 7] = -39S[6 8] = -21S[6 9] = (max-right = 9) -1S[610] = -8

mid =5

maximum subarray crossing mid is S[49] = 16

10

if high == lowReturn (low high A[low]) base case only one element

else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)

FIND-MAXIMUM-SUBARRAY (A low high)

Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)

2highlow

11

Analyzing time complexity

bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY

T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)

12

Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB

1

kj

n

kikij bac

An O(n3) time naive algorithm

SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj

return C

13

Divide-and-Conquer Algorithm

bull Assume that n is an exact power of 2

2221

1211

2221

1211

2221

1211 CC

CCC

BB

BBB

AA

AAA

2221

1211

2221

1211

2221

1211

BB

BB

AA

AA

CC

CC

(41)

14

Divide-and-Conquer Algorithm

2112111111 BABAC

2122112121 BABAC

2212121112 BABAC

2222122122 BABAC

A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)

= (n3)Computing A+B O(n2)

15

Strassenrsquos method

222131211222121 AASAASBBS

221162211511214 BBSAASBBS

211192221822127 AASBBSAAS

121110 BBS (42)

16

Strassenrsquos method

1111 SAP

2222 BSP

1133 BSP

4224 SAP

655 SSP

876 SSP

1097 SSP

624511 PPPPC

2112 PPC

4321 PPC

731522 PPPPC

(43)

(44)

17

Strassenrsquos divide-and-conquer algorithm

bull Step 1 Divide each of A B and C into four sub-matrices as in (41)

bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)

bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)

bull Step 4 Compute according to (44)

22211211 CCCC

18

Time complexity

T(n) = 7T(n2) + (n2)

= (nlg 7 ) (why)

= (n281)

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

4

Maximum-subarray problem

bull Input an array A[1n] of n numbersndash Assume that some of the numbers are negative

because this problem is trivial when all numbers are nonnegative

bull Output a nonempty subarray A[ij] having the largest sum S[i j] = ai + ai+1 + + aj

13 -3 -25 20 -3 -16 -23 18 20 -7 12 -5 -22 15 -4 7

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

A

maximum subarray

5

A brute-force solutionbull Examine all possible S[i j]bull Two implementationsndash compute each S[i j] in O(n) time O(n3) timendash compute each S[i j+1] from S[i j] in O(1) timendash (S[i i] = A[i] and S[i j+1] = S[i j] + A[j+1])ndash O(n2) time Ex i 1 2 3 4 5 6 A[i] 13 -3 -25 20 -3 -16

S[1 j] = 13 10 -15 5 2 -14 S[2 j] = -3 -28 -8 -11 -27

2

n

6

A divide-and-conquer solutionbull Possible locations of a maximum subarray A[ij] of

A[lowhigh] where mid = (low+high)2ndash entirely in A[lowmid] (low i j mid)ndash entirely in A[mid+1high] (mid lt i j high)ndash crossing the midpoint (low i mid lt j high)

low mid high

mid +1

entirely in A[lowmid] entirely in A[mid+1high]

crossing the midpoint

Possible locations of subarrays of A[lowhigh]

7

low mid high

mid +1

A[imid]

j

A[mid+1j]

i

A[ij] comprises two subarrays A[imid] and A[mid+1j]

8

FIND-MAX-CROSSING-SUBARRAY (A low mid high)

left-sum = - 1048576 Find a maximum subarray of the form A[imid] sum = 0for i = mid downto low sum = sum + A[i ] if sum gt left-sum left-sum = sum max-left = i right-sum = - 1048576 Find a maximum subarray of the form A[mid + 1 j ]sum =0for j = mid +1 to high sum = sum + A[j] if sum gt right-sum right-sum = sum max-right = j Return the indices and the sum of the two subarraysReturn (max-left max-right left-sum + right-sum)

9

1 2 3 4 5 6 7 8 9 10

13 -3 -25 20 -3 -16 -23 18 20 -7

Example

A

S[5 5] = -3S[4 5] = 17 (max-left = 4)S[3 5] = -8S[2 5] = -11S[1 5] = 2

mid =5

1 2 3 4 5 6 7 8 9 10

13 -3 -25 20 -3 -16 -23 18 20 -7A

S[6 6] = -16S[6 7] = -39S[6 8] = -21S[6 9] = (max-right = 9) -1S[610] = -8

mid =5

maximum subarray crossing mid is S[49] = 16

10

if high == lowReturn (low high A[low]) base case only one element

else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)

FIND-MAXIMUM-SUBARRAY (A low high)

Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)

2highlow

11

Analyzing time complexity

bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY

T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)

12

Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB

1

kj

n

kikij bac

An O(n3) time naive algorithm

SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj

return C

13

Divide-and-Conquer Algorithm

bull Assume that n is an exact power of 2

2221

1211

2221

1211

2221

1211 CC

CCC

BB

BBB

AA

AAA

2221

1211

2221

1211

2221

1211

BB

BB

AA

AA

CC

CC

(41)

14

Divide-and-Conquer Algorithm

2112111111 BABAC

2122112121 BABAC

2212121112 BABAC

2222122122 BABAC

A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)

= (n3)Computing A+B O(n2)

15

Strassenrsquos method

222131211222121 AASAASBBS

221162211511214 BBSAASBBS

211192221822127 AASBBSAAS

121110 BBS (42)

16

Strassenrsquos method

1111 SAP

2222 BSP

1133 BSP

4224 SAP

655 SSP

876 SSP

1097 SSP

624511 PPPPC

2112 PPC

4321 PPC

731522 PPPPC

(43)

(44)

17

Strassenrsquos divide-and-conquer algorithm

bull Step 1 Divide each of A B and C into four sub-matrices as in (41)

bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)

bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)

bull Step 4 Compute according to (44)

22211211 CCCC

18

Time complexity

T(n) = 7T(n2) + (n2)

= (nlg 7 ) (why)

= (n281)

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

5

A brute-force solutionbull Examine all possible S[i j]bull Two implementationsndash compute each S[i j] in O(n) time O(n3) timendash compute each S[i j+1] from S[i j] in O(1) timendash (S[i i] = A[i] and S[i j+1] = S[i j] + A[j+1])ndash O(n2) time Ex i 1 2 3 4 5 6 A[i] 13 -3 -25 20 -3 -16

S[1 j] = 13 10 -15 5 2 -14 S[2 j] = -3 -28 -8 -11 -27

2

n

6

A divide-and-conquer solutionbull Possible locations of a maximum subarray A[ij] of

A[lowhigh] where mid = (low+high)2ndash entirely in A[lowmid] (low i j mid)ndash entirely in A[mid+1high] (mid lt i j high)ndash crossing the midpoint (low i mid lt j high)

low mid high

mid +1

entirely in A[lowmid] entirely in A[mid+1high]

crossing the midpoint

Possible locations of subarrays of A[lowhigh]

7

low mid high

mid +1

A[imid]

j

A[mid+1j]

i

A[ij] comprises two subarrays A[imid] and A[mid+1j]

8

FIND-MAX-CROSSING-SUBARRAY (A low mid high)

left-sum = - 1048576 Find a maximum subarray of the form A[imid] sum = 0for i = mid downto low sum = sum + A[i ] if sum gt left-sum left-sum = sum max-left = i right-sum = - 1048576 Find a maximum subarray of the form A[mid + 1 j ]sum =0for j = mid +1 to high sum = sum + A[j] if sum gt right-sum right-sum = sum max-right = j Return the indices and the sum of the two subarraysReturn (max-left max-right left-sum + right-sum)

9

1 2 3 4 5 6 7 8 9 10

13 -3 -25 20 -3 -16 -23 18 20 -7

Example

A

S[5 5] = -3S[4 5] = 17 (max-left = 4)S[3 5] = -8S[2 5] = -11S[1 5] = 2

mid =5

1 2 3 4 5 6 7 8 9 10

13 -3 -25 20 -3 -16 -23 18 20 -7A

S[6 6] = -16S[6 7] = -39S[6 8] = -21S[6 9] = (max-right = 9) -1S[610] = -8

mid =5

maximum subarray crossing mid is S[49] = 16

10

if high == lowReturn (low high A[low]) base case only one element

else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)

FIND-MAXIMUM-SUBARRAY (A low high)

Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)

2highlow

11

Analyzing time complexity

bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY

T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)

12

Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB

1

kj

n

kikij bac

An O(n3) time naive algorithm

SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj

return C

13

Divide-and-Conquer Algorithm

bull Assume that n is an exact power of 2

2221

1211

2221

1211

2221

1211 CC

CCC

BB

BBB

AA

AAA

2221

1211

2221

1211

2221

1211

BB

BB

AA

AA

CC

CC

(41)

14

Divide-and-Conquer Algorithm

2112111111 BABAC

2122112121 BABAC

2212121112 BABAC

2222122122 BABAC

A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)

= (n3)Computing A+B O(n2)

15

Strassenrsquos method

222131211222121 AASAASBBS

221162211511214 BBSAASBBS

211192221822127 AASBBSAAS

121110 BBS (42)

16

Strassenrsquos method

1111 SAP

2222 BSP

1133 BSP

4224 SAP

655 SSP

876 SSP

1097 SSP

624511 PPPPC

2112 PPC

4321 PPC

731522 PPPPC

(43)

(44)

17

Strassenrsquos divide-and-conquer algorithm

bull Step 1 Divide each of A B and C into four sub-matrices as in (41)

bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)

bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)

bull Step 4 Compute according to (44)

22211211 CCCC

18

Time complexity

T(n) = 7T(n2) + (n2)

= (nlg 7 ) (why)

= (n281)

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

6

A divide-and-conquer solutionbull Possible locations of a maximum subarray A[ij] of

A[lowhigh] where mid = (low+high)2ndash entirely in A[lowmid] (low i j mid)ndash entirely in A[mid+1high] (mid lt i j high)ndash crossing the midpoint (low i mid lt j high)

low mid high

mid +1

entirely in A[lowmid] entirely in A[mid+1high]

crossing the midpoint

Possible locations of subarrays of A[lowhigh]

7

low mid high

mid +1

A[imid]

j

A[mid+1j]

i

A[ij] comprises two subarrays A[imid] and A[mid+1j]

8

FIND-MAX-CROSSING-SUBARRAY (A low mid high)

left-sum = - 1048576 Find a maximum subarray of the form A[imid] sum = 0for i = mid downto low sum = sum + A[i ] if sum gt left-sum left-sum = sum max-left = i right-sum = - 1048576 Find a maximum subarray of the form A[mid + 1 j ]sum =0for j = mid +1 to high sum = sum + A[j] if sum gt right-sum right-sum = sum max-right = j Return the indices and the sum of the two subarraysReturn (max-left max-right left-sum + right-sum)

9

1 2 3 4 5 6 7 8 9 10

13 -3 -25 20 -3 -16 -23 18 20 -7

Example

A

S[5 5] = -3S[4 5] = 17 (max-left = 4)S[3 5] = -8S[2 5] = -11S[1 5] = 2

mid =5

1 2 3 4 5 6 7 8 9 10

13 -3 -25 20 -3 -16 -23 18 20 -7A

S[6 6] = -16S[6 7] = -39S[6 8] = -21S[6 9] = (max-right = 9) -1S[610] = -8

mid =5

maximum subarray crossing mid is S[49] = 16

10

if high == lowReturn (low high A[low]) base case only one element

else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)

FIND-MAXIMUM-SUBARRAY (A low high)

Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)

2highlow

11

Analyzing time complexity

bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY

T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)

12

Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB

1

kj

n

kikij bac

An O(n3) time naive algorithm

SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj

return C

13

Divide-and-Conquer Algorithm

bull Assume that n is an exact power of 2

2221

1211

2221

1211

2221

1211 CC

CCC

BB

BBB

AA

AAA

2221

1211

2221

1211

2221

1211

BB

BB

AA

AA

CC

CC

(41)

14

Divide-and-Conquer Algorithm

2112111111 BABAC

2122112121 BABAC

2212121112 BABAC

2222122122 BABAC

A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)

= (n3)Computing A+B O(n2)

15

Strassenrsquos method

222131211222121 AASAASBBS

221162211511214 BBSAASBBS

211192221822127 AASBBSAAS

121110 BBS (42)

16

Strassenrsquos method

1111 SAP

2222 BSP

1133 BSP

4224 SAP

655 SSP

876 SSP

1097 SSP

624511 PPPPC

2112 PPC

4321 PPC

731522 PPPPC

(43)

(44)

17

Strassenrsquos divide-and-conquer algorithm

bull Step 1 Divide each of A B and C into four sub-matrices as in (41)

bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)

bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)

bull Step 4 Compute according to (44)

22211211 CCCC

18

Time complexity

T(n) = 7T(n2) + (n2)

= (nlg 7 ) (why)

= (n281)

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

7

low mid high

mid +1

A[imid]

j

A[mid+1j]

i

A[ij] comprises two subarrays A[imid] and A[mid+1j]

8

FIND-MAX-CROSSING-SUBARRAY (A low mid high)

left-sum = - 1048576 Find a maximum subarray of the form A[imid] sum = 0for i = mid downto low sum = sum + A[i ] if sum gt left-sum left-sum = sum max-left = i right-sum = - 1048576 Find a maximum subarray of the form A[mid + 1 j ]sum =0for j = mid +1 to high sum = sum + A[j] if sum gt right-sum right-sum = sum max-right = j Return the indices and the sum of the two subarraysReturn (max-left max-right left-sum + right-sum)

9

1 2 3 4 5 6 7 8 9 10

13 -3 -25 20 -3 -16 -23 18 20 -7

Example

A

S[5 5] = -3S[4 5] = 17 (max-left = 4)S[3 5] = -8S[2 5] = -11S[1 5] = 2

mid =5

1 2 3 4 5 6 7 8 9 10

13 -3 -25 20 -3 -16 -23 18 20 -7A

S[6 6] = -16S[6 7] = -39S[6 8] = -21S[6 9] = (max-right = 9) -1S[610] = -8

mid =5

maximum subarray crossing mid is S[49] = 16

10

if high == lowReturn (low high A[low]) base case only one element

else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)

FIND-MAXIMUM-SUBARRAY (A low high)

Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)

2highlow

11

Analyzing time complexity

bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY

T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)

12

Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB

1

kj

n

kikij bac

An O(n3) time naive algorithm

SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj

return C

13

Divide-and-Conquer Algorithm

bull Assume that n is an exact power of 2

2221

1211

2221

1211

2221

1211 CC

CCC

BB

BBB

AA

AAA

2221

1211

2221

1211

2221

1211

BB

BB

AA

AA

CC

CC

(41)

14

Divide-and-Conquer Algorithm

2112111111 BABAC

2122112121 BABAC

2212121112 BABAC

2222122122 BABAC

A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)

= (n3)Computing A+B O(n2)

15

Strassenrsquos method

222131211222121 AASAASBBS

221162211511214 BBSAASBBS

211192221822127 AASBBSAAS

121110 BBS (42)

16

Strassenrsquos method

1111 SAP

2222 BSP

1133 BSP

4224 SAP

655 SSP

876 SSP

1097 SSP

624511 PPPPC

2112 PPC

4321 PPC

731522 PPPPC

(43)

(44)

17

Strassenrsquos divide-and-conquer algorithm

bull Step 1 Divide each of A B and C into four sub-matrices as in (41)

bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)

bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)

bull Step 4 Compute according to (44)

22211211 CCCC

18

Time complexity

T(n) = 7T(n2) + (n2)

= (nlg 7 ) (why)

= (n281)

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

8

FIND-MAX-CROSSING-SUBARRAY (A low mid high)

left-sum = - 1048576 Find a maximum subarray of the form A[imid] sum = 0for i = mid downto low sum = sum + A[i ] if sum gt left-sum left-sum = sum max-left = i right-sum = - 1048576 Find a maximum subarray of the form A[mid + 1 j ]sum =0for j = mid +1 to high sum = sum + A[j] if sum gt right-sum right-sum = sum max-right = j Return the indices and the sum of the two subarraysReturn (max-left max-right left-sum + right-sum)

9

1 2 3 4 5 6 7 8 9 10

13 -3 -25 20 -3 -16 -23 18 20 -7

Example

A

S[5 5] = -3S[4 5] = 17 (max-left = 4)S[3 5] = -8S[2 5] = -11S[1 5] = 2

mid =5

1 2 3 4 5 6 7 8 9 10

13 -3 -25 20 -3 -16 -23 18 20 -7A

S[6 6] = -16S[6 7] = -39S[6 8] = -21S[6 9] = (max-right = 9) -1S[610] = -8

mid =5

maximum subarray crossing mid is S[49] = 16

10

if high == lowReturn (low high A[low]) base case only one element

else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)

FIND-MAXIMUM-SUBARRAY (A low high)

Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)

2highlow

11

Analyzing time complexity

bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY

T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)

12

Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB

1

kj

n

kikij bac

An O(n3) time naive algorithm

SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj

return C

13

Divide-and-Conquer Algorithm

bull Assume that n is an exact power of 2

2221

1211

2221

1211

2221

1211 CC

CCC

BB

BBB

AA

AAA

2221

1211

2221

1211

2221

1211

BB

BB

AA

AA

CC

CC

(41)

14

Divide-and-Conquer Algorithm

2112111111 BABAC

2122112121 BABAC

2212121112 BABAC

2222122122 BABAC

A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)

= (n3)Computing A+B O(n2)

15

Strassenrsquos method

222131211222121 AASAASBBS

221162211511214 BBSAASBBS

211192221822127 AASBBSAAS

121110 BBS (42)

16

Strassenrsquos method

1111 SAP

2222 BSP

1133 BSP

4224 SAP

655 SSP

876 SSP

1097 SSP

624511 PPPPC

2112 PPC

4321 PPC

731522 PPPPC

(43)

(44)

17

Strassenrsquos divide-and-conquer algorithm

bull Step 1 Divide each of A B and C into four sub-matrices as in (41)

bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)

bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)

bull Step 4 Compute according to (44)

22211211 CCCC

18

Time complexity

T(n) = 7T(n2) + (n2)

= (nlg 7 ) (why)

= (n281)

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

9

1 2 3 4 5 6 7 8 9 10

13 -3 -25 20 -3 -16 -23 18 20 -7

Example

A

S[5 5] = -3S[4 5] = 17 (max-left = 4)S[3 5] = -8S[2 5] = -11S[1 5] = 2

mid =5

1 2 3 4 5 6 7 8 9 10

13 -3 -25 20 -3 -16 -23 18 20 -7A

S[6 6] = -16S[6 7] = -39S[6 8] = -21S[6 9] = (max-right = 9) -1S[610] = -8

mid =5

maximum subarray crossing mid is S[49] = 16

10

if high == lowReturn (low high A[low]) base case only one element

else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)

FIND-MAXIMUM-SUBARRAY (A low high)

Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)

2highlow

11

Analyzing time complexity

bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY

T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)

12

Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB

1

kj

n

kikij bac

An O(n3) time naive algorithm

SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj

return C

13

Divide-and-Conquer Algorithm

bull Assume that n is an exact power of 2

2221

1211

2221

1211

2221

1211 CC

CCC

BB

BBB

AA

AAA

2221

1211

2221

1211

2221

1211

BB

BB

AA

AA

CC

CC

(41)

14

Divide-and-Conquer Algorithm

2112111111 BABAC

2122112121 BABAC

2212121112 BABAC

2222122122 BABAC

A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)

= (n3)Computing A+B O(n2)

15

Strassenrsquos method

222131211222121 AASAASBBS

221162211511214 BBSAASBBS

211192221822127 AASBBSAAS

121110 BBS (42)

16

Strassenrsquos method

1111 SAP

2222 BSP

1133 BSP

4224 SAP

655 SSP

876 SSP

1097 SSP

624511 PPPPC

2112 PPC

4321 PPC

731522 PPPPC

(43)

(44)

17

Strassenrsquos divide-and-conquer algorithm

bull Step 1 Divide each of A B and C into four sub-matrices as in (41)

bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)

bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)

bull Step 4 Compute according to (44)

22211211 CCCC

18

Time complexity

T(n) = 7T(n2) + (n2)

= (nlg 7 ) (why)

= (n281)

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

10

if high == lowReturn (low high A[low]) base case only one element

else mid = (left-low left-high left-sum) = FIND-MAXIMUM-SUBARRAY(A low mid) (right-low right-high right-sum) = FIND-MAXIMUM-SUBARRAY(A mid + 1 high) (cross-low cross-high cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A low mid high) if left-sum ≧ right-sum and left-sum ≧ cross-sum return (left-low left-high left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low right-high right-sum) else return (cross-low cross-high cross-sum)

FIND-MAXIMUM-SUBARRAY (A low high)

Initial call FIND-MAXIMUM-SUBARRAY (A 1 n)

2highlow

11

Analyzing time complexity

bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY

T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)

12

Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB

1

kj

n

kikij bac

An O(n3) time naive algorithm

SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj

return C

13

Divide-and-Conquer Algorithm

bull Assume that n is an exact power of 2

2221

1211

2221

1211

2221

1211 CC

CCC

BB

BBB

AA

AAA

2221

1211

2221

1211

2221

1211

BB

BB

AA

AA

CC

CC

(41)

14

Divide-and-Conquer Algorithm

2112111111 BABAC

2122112121 BABAC

2212121112 BABAC

2222122122 BABAC

A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)

= (n3)Computing A+B O(n2)

15

Strassenrsquos method

222131211222121 AASAASBBS

221162211511214 BBSAASBBS

211192221822127 AASBBSAAS

121110 BBS (42)

16

Strassenrsquos method

1111 SAP

2222 BSP

1133 BSP

4224 SAP

655 SSP

876 SSP

1097 SSP

624511 PPPPC

2112 PPC

4321 PPC

731522 PPPPC

(43)

(44)

17

Strassenrsquos divide-and-conquer algorithm

bull Step 1 Divide each of A B and C into four sub-matrices as in (41)

bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)

bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)

bull Step 4 Compute according to (44)

22211211 CCCC

18

Time complexity

T(n) = 7T(n2) + (n2)

= (nlg 7 ) (why)

= (n281)

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

11

Analyzing time complexity

bull FIND-MAX-CROSSING-SUBARRAY (n) where n = high low + 1bull FIND-MAXIMUM-SUBARRAY

T(n) = 2T(n2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)

12

Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB

1

kj

n

kikij bac

An O(n3) time naive algorithm

SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj

return C

13

Divide-and-Conquer Algorithm

bull Assume that n is an exact power of 2

2221

1211

2221

1211

2221

1211 CC

CCC

BB

BBB

AA

AAA

2221

1211

2221

1211

2221

1211

BB

BB

AA

AA

CC

CC

(41)

14

Divide-and-Conquer Algorithm

2112111111 BABAC

2122112121 BABAC

2212121112 BABAC

2222122122 BABAC

A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)

= (n3)Computing A+B O(n2)

15

Strassenrsquos method

222131211222121 AASAASBBS

221162211511214 BBSAASBBS

211192221822127 AASBBSAAS

121110 BBS (42)

16

Strassenrsquos method

1111 SAP

2222 BSP

1133 BSP

4224 SAP

655 SSP

876 SSP

1097 SSP

624511 PPPPC

2112 PPC

4321 PPC

731522 PPPPC

(43)

(44)

17

Strassenrsquos divide-and-conquer algorithm

bull Step 1 Divide each of A B and C into four sub-matrices as in (41)

bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)

bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)

bull Step 4 Compute according to (44)

22211211 CCCC

18

Time complexity

T(n) = 7T(n2) + (n2)

= (nlg 7 ) (why)

= (n281)

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

12

Matrix multiplicationbull Input two n n matrices A and Bbull Output C = AB

1

kj

n

kikij bac

An O(n3) time naive algorithm

SQUARE-MATRIX-MULTIPLY(A B)n Arowslet C be an n n matrixfor i 1 to n for j 1 to n cij 0 for k 1 to n cij cij + aikbkj

return C

13

Divide-and-Conquer Algorithm

bull Assume that n is an exact power of 2

2221

1211

2221

1211

2221

1211 CC

CCC

BB

BBB

AA

AAA

2221

1211

2221

1211

2221

1211

BB

BB

AA

AA

CC

CC

(41)

14

Divide-and-Conquer Algorithm

2112111111 BABAC

2122112121 BABAC

2212121112 BABAC

2222122122 BABAC

A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)

= (n3)Computing A+B O(n2)

15

Strassenrsquos method

222131211222121 AASAASBBS

221162211511214 BBSAASBBS

211192221822127 AASBBSAAS

121110 BBS (42)

16

Strassenrsquos method

1111 SAP

2222 BSP

1133 BSP

4224 SAP

655 SSP

876 SSP

1097 SSP

624511 PPPPC

2112 PPC

4321 PPC

731522 PPPPC

(43)

(44)

17

Strassenrsquos divide-and-conquer algorithm

bull Step 1 Divide each of A B and C into four sub-matrices as in (41)

bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)

bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)

bull Step 4 Compute according to (44)

22211211 CCCC

18

Time complexity

T(n) = 7T(n2) + (n2)

= (nlg 7 ) (why)

= (n281)

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

13

Divide-and-Conquer Algorithm

bull Assume that n is an exact power of 2

2221

1211

2221

1211

2221

1211 CC

CCC

BB

BBB

AA

AAA

2221

1211

2221

1211

2221

1211

BB

BB

AA

AA

CC

CC

(41)

14

Divide-and-Conquer Algorithm

2112111111 BABAC

2122112121 BABAC

2212121112 BABAC

2222122122 BABAC

A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)

= (n3)Computing A+B O(n2)

15

Strassenrsquos method

222131211222121 AASAASBBS

221162211511214 BBSAASBBS

211192221822127 AASBBSAAS

121110 BBS (42)

16

Strassenrsquos method

1111 SAP

2222 BSP

1133 BSP

4224 SAP

655 SSP

876 SSP

1097 SSP

624511 PPPPC

2112 PPC

4321 PPC

731522 PPPPC

(43)

(44)

17

Strassenrsquos divide-and-conquer algorithm

bull Step 1 Divide each of A B and C into four sub-matrices as in (41)

bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)

bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)

bull Step 4 Compute according to (44)

22211211 CCCC

18

Time complexity

T(n) = 7T(n2) + (n2)

= (nlg 7 ) (why)

= (n281)

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

14

Divide-and-Conquer Algorithm

2112111111 BABAC

2122112121 BABAC

2212121112 BABAC

2222122122 BABAC

A straightforward divide-and-conquer algorithm T(n) = 8T(n2) + (n2)

= (n3)Computing A+B O(n2)

15

Strassenrsquos method

222131211222121 AASAASBBS

221162211511214 BBSAASBBS

211192221822127 AASBBSAAS

121110 BBS (42)

16

Strassenrsquos method

1111 SAP

2222 BSP

1133 BSP

4224 SAP

655 SSP

876 SSP

1097 SSP

624511 PPPPC

2112 PPC

4321 PPC

731522 PPPPC

(43)

(44)

17

Strassenrsquos divide-and-conquer algorithm

bull Step 1 Divide each of A B and C into four sub-matrices as in (41)

bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)

bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)

bull Step 4 Compute according to (44)

22211211 CCCC

18

Time complexity

T(n) = 7T(n2) + (n2)

= (nlg 7 ) (why)

= (n281)

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

15

Strassenrsquos method

222131211222121 AASAASBBS

221162211511214 BBSAASBBS

211192221822127 AASBBSAAS

121110 BBS (42)

16

Strassenrsquos method

1111 SAP

2222 BSP

1133 BSP

4224 SAP

655 SSP

876 SSP

1097 SSP

624511 PPPPC

2112 PPC

4321 PPC

731522 PPPPC

(43)

(44)

17

Strassenrsquos divide-and-conquer algorithm

bull Step 1 Divide each of A B and C into four sub-matrices as in (41)

bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)

bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)

bull Step 4 Compute according to (44)

22211211 CCCC

18

Time complexity

T(n) = 7T(n2) + (n2)

= (nlg 7 ) (why)

= (n281)

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

16

Strassenrsquos method

1111 SAP

2222 BSP

1133 BSP

4224 SAP

655 SSP

876 SSP

1097 SSP

624511 PPPPC

2112 PPC

4321 PPC

731522 PPPPC

(43)

(44)

17

Strassenrsquos divide-and-conquer algorithm

bull Step 1 Divide each of A B and C into four sub-matrices as in (41)

bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)

bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)

bull Step 4 Compute according to (44)

22211211 CCCC

18

Time complexity

T(n) = 7T(n2) + (n2)

= (nlg 7 ) (why)

= (n281)

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

17

Strassenrsquos divide-and-conquer algorithm

bull Step 1 Divide each of A B and C into four sub-matrices as in (41)

bull Step 2 Create 10 matrices S1 S2 hellip S10 as in (42)

bull Steep 3 Recursively compute P1 P2 hellip P7 as in (43)

bull Step 4 Compute according to (44)

22211211 CCCC

18

Time complexity

T(n) = 7T(n2) + (n2)

= (nlg 7 ) (why)

= (n281)

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

18

Time complexity

T(n) = 7T(n2) + (n2)

= (nlg 7 ) (why)

= (n281)

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

19

Discussion

bull Strassenrsquos method is largely of theoretical interest for n 45

bull Strassenrsquos method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8)

bull It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

20

Discussion

bull We can improve Strassenrsquos algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications where k gt 2 The time is T(n) = qT(nk) + θ(n2)

bull A trivial lower bound for matrix multiplication is (n2) The current best upper bound known is O(n2376)

bull Open problemsndash Can the upper bound O(n2376) be improvedndash Can the lower bound (n2) be improved

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

21

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull eg to show upper bound we find constants c and

n0 such that T(n) c f(n) for n = n0 n0+1 n0+2 hellip

2n

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

22

Substitution Method(if we know the answer)

How to solve this T(n) = 2T( ) + n with T(1) = 1

1 Make a guess eg T(n) = O(n log n)

2 Show it by induction bull Firstly T(2) = 4 T(3) = 5

We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay

bull Other Cases

2n

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

23

Substitution Method(if we know the answer)

bull Induction CaseAssume the guess is true for all n = 2 3hellip kFor n = k+1 we have

T(n) = 2T( ) + n

= cn lg n ndash cn + n cn log n

Induction case is true

2n

nnnc 2lg22

nncn 2lg

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

24

Substitution Method(if we know the answer)

Q How did we know the value of c and n0

A If induction works the induction case must be

correct c ge 1 Then we find that by setting c = 2 our guess is

correct as soon as n0 = 2

Alternatively we can also use c = 15 Then we just need a larger n0 = 4

(What will be the new base cases Why)

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

25

Substitution Method(New Challenge)

How to solve this

1 Make a guess (T(n) = O(n)) and 2 Show T(n) le cn by induction ndash What will happen in induction case

1)1( 1)2()2()( TnTnTnT

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

26

Substitution Method(New Challenge)

Induction Case (assume guess is true for some base cases)

This term is not what we

want hellip

1

122

1)2()2()(

cn

ncnc

nTnTnT

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

27

Substitution Method(New Challenge)

bull The 1st attempt was not working because our guess for T(n) was a bit ldquolooserdquo

Recall Induction may become easier if we prove a ldquostrongerrdquo statement

2nd Attempt Refine our statementTry to show T(n) le cn - b instead

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

28

Substitution Method(New Challenge)

bcn

bncbnc

nTnTnT

122

1)2()2()(

Induction Case

It remains to find c and n0 and prove the base case(s) which is relatively easyCan you prove T(n) le cn2 by induction

We get the desired term (when b 1)

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

29

Avoiding Pitfalls

bull For T(n) = 2T( ) + n we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing

)(

)2(2)(

nO

ncn

nncnT

2n

Wrong

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

30

Substitution Method(New Challenge 2)

How to solve this

T(n) = 2T( ) + lg n n

Hint Change variable Set m = lg n

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

31

Substitution Method(New Challenge 2)

Set m = lg n we get

T(2m) = 2T(2m2) + m

Next set S(m) = T(2m) = T(n)

S(m) = 2S(m2) + m

We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

32

Recursion Tree Method( Nothing Specialhellip Very Useful )

How to solve this T(n) = 2T(n2) + n2 with T(1) = 1

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

33

Recursion Tree Method( Nothing Specialhellip Very Useful )

Expanding the terms we get

T(n) = n2 + 2T(n2) = n2 + 2n24 + 4T(n4) = n2 + 2n24 + 4n216 + 8T(n8) = =

= (n2) + (n) = (n2)

)1(2)21( lg21lg

0Tn nn

k

k

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

34

Recursion Tree Method( Recursion Tree View )

We can express the previous recurrence by

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

35

Further expressing gives us

This term is from T(n2)

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

36

Recursion Tree Method( New Challenge )

How to solve this T(n) = T(n3) + T(2n3) + n with T(1) = 1

What will be the recursion tree view

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

37

The corresponding recursion tree view is

The depth of the tree is log32 n Why

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

38

Master Method( Save our effort )

When the recurrence is in a special form we can apply the Master Theorem to solve the recurrence immediately

The Master Theorem has 3 cases hellip

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

39

Master Theorem

Theorem (Case 1)

If f(n) = O(nlogb a - ) for some constant 0

then T(n) = (nlogb a)

Let T(n) = aT(nb) + f(n) with a 1 and b 1 are constants

wherewe interpret nb to mean either nb or nb

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

40

Theorem (Case 2)

If f(n) = (nlogb a)

then T(n) = (nlogb a lg n)

Theorem (Case 3)

If f(n) = (nlogb a + ) for some constant 0

and if af(nb) c f(n) for some constant c 1 and all sufficiently large n

then T(n) = (f(n))

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

41

Master Theorem 1 Solve T(n) = 9T(n3) + n (case 1)

2 Solve T(n) = T(2n3) + 1 (case 2)

3 Solve T(n) = 3T(n4) + nlgn (case 3)

4 How about this T(n) = 2T(n2) + n lg n

5 T(n) = 8T(n2) + n2 T(n) = 8T(n2) + n

6 T(n) = 7T(n2) + n2 T(n) = 7T(n2) + 1

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4

42

Homework

bull Exercise 41-3 (Programming) (due Oct 17)bull Practice at home 41-5 42-1 42-7

bull Exercise 43-6 44-6 45-1 (due Oct 19)bull Problem 41 (a g) (due Oct19)bull Practice at home 43-7 44-8 44-7 45-4