Divide and ConquerFaculty Name: Ruhi Fatima

Topics Covered• Divide and Conquer• Matrix multiplication• Recurrence

Divide and Conquer• Divide the problem into a number of subproblems that are smaller

instances of the same problem.

• Conquer the subproblems by solving them recursively. If the subproblem sizes are small enough, however, just solve the subproblems in a straightforward manner.

• Combine the solutions of the subproblems into the solution for the original problem.

• When the subproblems are large enough to solve recursively, we call that the recursive case. Once the subproblems become small enough that we no longer recursive, we say that the recursion “bottoms out” and that we have gotten down to the base case.

• Let A, B and C be n n matrices C = AB C(i, j) = A(i, k)B(k, j)

• The straightforward method (Brute Force Method) to perform a matrix multiplication requires 8 multiplications and 4 additions resulting O(n3) time complexity.

Divide-and-conquer approach• C = AB C11 = A11 B11 + A12 B21 C12 = A11B12 + A12 B22 C21 = A21 B11 + A22 B21 C22 = A21 B12 + A22 B22 • Time complexity: (# of additions : n2)

We get T(n) = O(n3)

1 k n

Matrix multiplication

C11 C12 = A11 A12 B11 B12 C21 C22 = A21 A22 B21 B22

T(n) = b 8T(n/2)+cn2 , n 2 , n > 2

Divide and Conquer ( Cont..)

• Derive a recurrence to characterize the running time of SQUAREMATRIX-MULTIPLY-RECURSIVE.

• Let T(n) be the time to multiply two n x n matrices using this procedure.

• In the base case, when n = 1, we perform just the one scalar multiplication in line 4, and so

( Cont..)• The recursive case occurs when n > 1. • In lines 6–9, we recursively call SQUARE-MATRIX-MULTIPLY-

RECURSIVE a total of eight times. • Because each recursive call multiplies two n/2 x n/2 matrices, thereby

contributing T (n/2) to the overall running time, the time taken by all eight recursive calls is 8T(n/2).

• Also must account for the four matrix additions in lines 6–9. • Each of these matrices contains n2/4 entries, and so each of the four

matrix additions takes Θ(n2) time. • The total time for the recursive case, therefore, is the sum of the

partitioning time, the time for all the recursive calls, and the time to add the matrices resulting from the recursive calls:

• Combining equations (4.15) and (4.16) gives us the recurrence for the running time of SQUARE-MATRIX-MULTIPLY-RECURSIVE:

1. Divide the input matrices A and B and output matrix C into n/2 x n/2 submatrices. This step takes Θ(1) time by index calculation, just as in SQUARE-MATRIX-MULTIPLY.2. Create 10 matrices S1, S2, ….,S10, each of which is n/2 x n/2 and is the sum or difference of two matrices created in step 1. All 10 matrices can be created in Θ(n2) time.3. Using the submatrices created in step 1 and the 10 matrices created in step 2, recursively compute seven matrix products P1, P2, ….,P7. Each matrix Pi is n/2 x n/2 .4. Compute the desired submatrices C11,C12, C21, C22 of the result matrix C by adding and subtracting various combinations of the Pi matrices. We can compute all four submatrices in Θ(n2) time.

Strassen’s matrix multiplication Method

• P1 = A11 S1 = A11B12 - A11B22 P2 = S2 B22 = A11B22 + A12B22 P3 = S3 B11 = A21B11 + A22B11 P4 = A22 S4 = A22 B21 – A22B11 P5 = S5 S6 = A11 B11 + A11B22 + A22 B11 + A22B22 P6 = S7 S8 = A12 B21 + A12B22 - A22 B21 - A22B22 P7 = S9 S10 = A11 B11 + A11B12 - A21 B11 - A21B12

• C11 = P5 + P4 - P2 + P6C12 = P1 + P2C21 = P3 + P4C22 = P5 + P1 - P3 - P7

C11 = A11 B11 + A12 B21

C12 = A11B12 + A12 B22

C21 = A21 B11 + A22 B21

C22 = A21 B12 + A22 B22

Cont… S1 = B12 - B22 , S2 = A11 +

A12 S3 = A21 + A22 , S4 = B21 -

B11

S5 = A11 + A22 , S6 = B11 + B22

S7 = A12 - A22 , S8 = B21 + B22

S9 = A11 - A21 , S10= B11 + B12

• 7 multiplications and 18 additions or subtractions• Time complexity:

T(n) = b 7T(n/2)+an2 , n 2 , n > 2

)(O )(O

)(

constanta is ,7)(

)1(7))()(1(

)4/(7

)4/(7)((7

)2/(7)(

81.27log

7log4log7log4log7loglog472

loglog472

1472

47

472

22472

22

2

2

2

222222

22

nn

ncnncn

ccn

Tan

nTanan

nTaan

nTannT

n

nn

kk

n

Time Complexity of Strassens

• A recurrence is an equation or inequality that describes a function in terms of its value on smaller inputs.

EXAMPLE : Compute the factorial function F(n) = n! for an arbitrary nonnegative integer n. Since n!= 1 . . . . . (n − 1) . n = (n − 1)! . n for n ≥ 1 and 0!= 1 by definition, we can compute F(n) = F(n − 1) . n with the following recursive algorithm.

ALGORITHM F(n)//Computes n! recursively//Input: A nonnegative integer n //Output: The value of n!

if n = 0 return 1else return F(n − 1) ∗ n

• The basic operation of the algorithm is multiplication, whose number of executions we denote M(n). Since the function F(n) is computed according to the formula F(n) = F(n − 1) . n for n > 0, F(0) = 1.

M(n) = M(n − 1) + 1 for n > 0, M(0) = 0.• So, two recursively defined functions are dealed. The first is the factorial

function F(n) itself, the second is the number of multiplications M(n) needed to compute F(n) by the recursion.

Recurrence

There are three methods for solving recurrences• Substitution method, guess a bound and then use mathematical

induction to prove our guess correct. • Recursion-tree method converts the recurrence into a tree whose

nodes represent the costs incurred at various levels of the recursion. We use techniques for bounding summations to solve the recurrence.

• Master method provides bounds for recurrences of the formT(n)= aT(n/b) + f(n)

where a≥1, b >1, and f (n) is a given function. A recurrence of the form in equation characterizes a divide and-conquer algorithm that creates a subproblems, each of which is 1/b the size of the original problem, and in which the divide and combine steps together take f (n) time

Methods for solving recurrences

• The substitution method for solving recurrences (divide and conquer) comprises two steps:1. Guess the form of the solution.2. Use mathematical induction to find the constants and show that the solution works.

• Substitute the guessed solution for the function when applying the inductive hypothesis to smaller values; hence the name “substitution method.” This method is powerful, but we must be able to guess the form of the answer in order to apply it.

• Substitution method to establish either upper or lower bounds on a recurrence.

Substitution Method

• Example: Determine an upper bound on the recurrence

• We guess that the solution is T(n) =O(n lg n). • The substitution method requires us to prove that T(n)≤cn lg n

for an appropriate choice of the constant c > 0. • Substituting into the recurrence yields

Cont…

• In a recursion tree, each node represents the cost of a single subproblem somewhere in the set of recursive function invocations. We sum the costs within each level of the tree to obtain a set of per-level costs, and then we sum all the per-level costs to determine the total cost of all levels of the recursion.

• For example, let us see how a recursion tree would provide a good guess for the recurrence

• We start by focusing on finding an upper bound for the solution. Because we know that floors and ceilings usually do not matter when solving recurrences (here’s an example of sloppiness that we can tolerate), we create a recursion tree for the recurrence, having written out the implied constant coefficient c > 0.

Recursion Tree

Example of recursion treeSolve T(n) = T(n/4) + T(n/2) + n2:

(n/16)2 (n/8)2 (n/8)2 (n/4)2

(n/4)2

Q(1)

…

2165 n

2n

225625 n

1 31652

165

1652 n

…

Total =

= Q(n2)

n2

(n/2)2

geometric series

1

111

2

xxxxxn

n

for x ¹ 1

1

11 2

xxx

for |x| < 1

Appendix: geometric series

• The master method provides a “cookbook” method for solving recurrences of the form

T (n) = aT (n/b) + f (n) ; where a ≥ 1 and b > 1 are constants and f (n) is an asymptotically positive function.• Master method depend on the Master Theorem

Master Theorem

• Example 1: Solve the recurrence T(n) = 9T(n/3) +n.Solution: In the given recurrence a = 9, b = 3, f(n) = n, and thus Since f(n) = O (, where So, by applying case 1 we get T(n)=.

• Example 2: Consider T( n) = T ( 2n/3) +1Solution: In the above a = 1, b = 3/2, f(n) = 1, and . So, case 2 is to be applied, which results the solution to the recurrence is T(n) =Θ (lg n).

• Example 3: Consider the recurrence T(n) = 3 T(n/4) + n lg n.Solution: In the above a = 3, b = 4, f(n) = n lg n, and . Since f(n) = Ω () , where ϵ ≈ 0.2, case 3 applies if we can show that the regularity condition holds for f( n). For sufficiently large n, we have that af( n/b ) = 3(n/4)lg (n/4) ≤ (3/4) lg (n/4) ≤ (3/4)n lg n = c f(n) for c=3/4.

Cont…

• Example 4: Solve the recurrence T(n) = 2 T(n/2) + n lg n.Solution: The master method does not apply to the above recurrence.Even though it appears to have the proper form: a = 2, b = 2, f(n)=nlgn, and = n. It might be mistakenly as case 3, since f (n) = n lg n is asymptotically larger than = n. The problem is that it is not polynomially larger. The ratio f (n ) / = n lg n/n = lg n is asymptotically less than n for any positive constant .Consequently, the recurrence falls into the gap between case 2 and case 3. Example 5: Let’s use the master method to solve the recurrences T (n)= 2T(n /2 )+ (n) , characterizes the running times of the divide-and-conquer algorithm.Here, we have a = 2, b = 2, f (n) = (n) and thus we have that = = n. Case 2 applies, since f (n) = (n), and so we have the solution T(n) = (n lg n).

Cont…

Example 5: Let’s use the master method to solve the recurrences T (n)= 8T(n /2 )+ (n2) , describes the running times of the divide-and-conquer algorithm for matrix multiplication. Now we have a = 8, b = 2, and f (n) = (n2), and so . Since is polynomially larger than f (n) (i.e., f (n) = O(n3-ϵ) for ϵ = 1), case 1 applies, and T (n) = (n3).

Example 6: Consider recurrence T (n)= 7T(n /2 )+ (n2), which describes the running time of Strassen’s algorithm. Here, we have a =7, b = 2, f (n) = (n2), and thus . Rewriting as lg 7 and recalling that 2:80 < lg7 < 2:81, we see that f (n) = O(nlg 7-ϵ) for ϵ = 0.8.Again, case 1 applies, and we have the solution T (n) = (nlg 7).

Cont…

• The proof appears in two parts. • The first part analyzes the master recurrence, under the simplifying

assumption that T (n) is defined only on exact powers of b > 1, i.e, for n = 1, b, b2, .....

• The second part shows how to extend the analysis to all positive integers n; it applies mathematical technique to the problem of handling floors and ceilings.

• The proof for exact powers :The first part of the proof of the master theorem analyzes the recurrence T (n) = aT(n/b) + f(n) , under the assumption that n is an exact power of b > 1, where b need not be an integer.

• The analysis is broken into three lemmas. • The first reduces the problem of solving the master recurrence to

the problem of evaluating an expression that contains a summation. • The second determines bounds on this summation. • The third lemma puts the first two together to prove a version of the

master theorem for the case in which n is an exact power of b.

Proof of Master Theorem

• Lemma 1Proof of Master Theorem