Divide and Conquer Strategy

2

General Concept of Divide & Conquer

Given a function to compute on n inputs, the divide-and-conquer strategy consists of: splitting the inputs into k distinct subsets, 1kn,

yielding k subproblems. solving these subproblems combining the subsolutions into solution of the whole. if the subproblems are relatively large, then

divide_Conquer is applied again. if the subproblems are small, the are solved without

splitting.

3

The Divide and Conquer Algorithm

Divide_Conquer(problem P){

if Small(P) return S(P);else { divide P into smaller instances P1, P2, …, Pk, k1; Apply Divide_Conquer to each of these subproblems; return Combine(Divide_Conque(P1), Divide_Conque(P2),…, Divide_Conque(Pk));}

}

4

Divde_Conquer recurrence relation

The computing time of Divde_Conquer is

T(n) is the time for Divde_Conquer on any input size n. g(n) is the time to compute the answer directly (for

small inputs) f(n) is the time for dividing P and combining the

solutions.

)()(...)()(

)()(

21 nfnTnTnT

ngnT

k

n small

otherwise

5

Three Steps of The Divide and Conquer Approach

The most well known algorithm design strategy:1. Divide the problem into two or more smaller

subproblems.

2. Conquer the subproblems by solving them recursively.

3. Combine the solutions to the subproblems into the solutions for the original problem.

6

A Typical Divide and Conquer Case

subproblem 2 of size n/2

subproblem 1 of size n/2

a solution to subproblem 1

a solution tothe original problem

a solution to subproblem 2

a problem of size n

7

An Example: Calculating a0 + a1 + … + an-1

Efficiency: (for n = 2k)

A(n) = 2A(n/2) + 1, n >1

A(1) = 0;

ALGORITHM RecursiveSum(A[0..n-1])//Input: An array A[0..n-1] of orderable elements//Output: the summation of the array elementsif n > 1

return ( RecursiveSum(A[0.. n/2 – 1]) + RecursiveSum(A[n/2 .. n– 1]) )

8

General Divide and Conquer recurrence

The Master Theorem

T(n) = aT(n/b) + f (n), where f (n) ∈ Θ(nk) 1. a < bk T(n) ∈ Θ(nk) 2. a = bk T(n) ∈ Θ(nk lg n ) 3. a > bk T(n) ∈ Θ(nlog b a)

the time spent on solving a subproblem of size n/b.

the time spent on dividing the problem

into smaller ones and combining their solutions.

9

Divide and Conquer Other Examples

Sorting: mergesort and quicksort

Binary search

Matrix multiplication-Strassen’s algorithm

10

The Divide, Conquer and Combine Steps in Quicksort

Divide: Partition array A[l..r] into 2 subarrays, A[l..s-1] and A[s+1..r] such that each element of the first array is ≤A[s] and each element of the second array is ≥ A[s]. (computing the index of s is part of partition.) Implication: A[s] will be in its final position in the

sorted array. Conquer: Sort the two subarrays A[l..s-1] and

A[s+1..r] by recursive calls to quicksort Combine: No work is needed, because A[s] is already

in its correct place after the partition is done, and the two subarrays have been sorted.

11

Quicksort Select a pivot whose value we are going to divide

the sublist. (e.g., p = A[l]) Rearrange the list so that it starts with the pivot

followed by a ≤ sublist (a sublist whose elements are all smaller than or equal to the pivot) and a ≥ sublist (a sublist whose elements are all greater than or equal to the pivot ) (See algorithm Partition in section 4.2)

Exchange the pivot with the last element in the first sublist(i.e., ≤ sublist) – the pivot is now in its final position

Sort the two sublists recursively using quicksort.

p

A[i]≤p A[i] ≥ p

12

The Quicksort Algorithm

ALGORITHM Quicksort(A[l..r])//Sorts a subarray by quicksort//Input: A subarray A[l..r] of A[0..n-1],defined by its left

and right indices l and r//Output: The subarray A[l..r] sorted in nondecreasing

orderif l < r

s Partition (A[l..r]) // s is a split positionQuicksort(A[l..s-1])Quicksort(A[s+1..r]

13

The Partition Algorithm

ALGORITHM Partition (A[l ..r])//Partitions a subarray by using its first element as a pivot//Input: A subarray A[l..r] of A[0..n-1], defined by its left and right

indices l and r (l < r)//Output: A partition of A[l..r], with the split position returned as this

function’s valueP A[l]i l; j r + 1;Repeat

repeat i i + 1 until A[i]>=p //left-right scanrepeat j j – 1 until A[j] <= p//right-left scanif (i < j) //need to continue with the scan

swap(A[i], a[j])until i >= j //no need to scanswap(A[l], A[j])return j

14

Quicksort Example

15 22 13 27 12 10 20 25

15

Efficiency of Quicksort

Based on whether the partitioning is balanced. Best case: split in the middle — Θ( n log n)

T(n) = 2T(n/2) + Θ(n) //2 subproblems of size n/2 each

Worst case: sorted array! — Θ( n2) T(n) = T(n-1) + n+1 //2 subproblems of size 0 and n-1 respectively

Average case: random arrays — Θ( n log n)

16

Binary Search – an Iterative Algorithm

ALGORITHM BinarySearch(A[0..n-1], K)//Implements nonrecursive binary search//Input: An array A[0..n-1] sorted in ascending order and // a search key K//Output: An index of the array’s element that is equal to K// or –1 if there is no such elementl 0, r n – 1while l r do //l and r crosses over can’t find K.

m (l + r) / 2if K = A[m] return m //the key is foundelse

if K < A[m] r m – 1 //the key is on the left half of the arrayelse l m + 1 // the key is on the right half of the

arrayreturn -1

17

Strassen’s Matrix Multiplication

Brute-force algorithmc00 c01 a00 a01 b00 b01

= *

c10 c11 a10 a11 b10 b11

a00 * b00 + a01 * b10 a00 * b01 + a01 * b11

=

a10 * b00 + a11 * b10 a10 * b01 + a11 * b11

8 multiplications

4 additions

Efficiency class: (n3)

Strassen’s Matrix Multiplication

Strassen’s Algorithm (two 2x2 matrices)c00 c01 a00 a01 b00 b01

= *c10 c11 a10 a11 b10 b11

m1 + m4 - m5 + m7 m3 + m5

= m2 + m4 m1 + m3 - m2 + m6

m1 = (a00 + a11) * (b00 + b11) m2 = (a10 + a11) * b00 m3 = a00 * (b01 - b11) m4 = a11 * (b10 - b00) m5 = (a00 + a01) * b11 m6 = (a10 - a00) * (b00 + b01) m7 = (a01 - a11) * (b10 + b11)

7 multiplications

18 additions

19

Strassen’s Matrix Multiplication

Two nxn matrices, where n is a power of two (What about the situations where n is not a power of two?)

C00 C01 A00 A01 B00 B01

= *

C10 C11 A10 A11 B10 B11

M1 + M4 - M5 + M7 M3 + M5

=

M2 + M4 M1 + M3 - M2 + M6

20

Submatrices:

M1 = (A00 + A11) * (B00 + B11)

M2 = (A10 + A11) * B00

M3 = A00 * (B01 - B11)

M4 = A11 * (B10 - B00)

M5 = (A00 + A01) * B11

M6 = (A10 - A00) * (B00 + B01)

M7 = (A01 - A11) * (B10 + B11)

21

Efficiency of Strassen’s Algorithm

If n is not a power of 2, matrices can be padded with rows and columns with zeros

Number of multiplications

Number of additions

Other algorithms have improved this result, but are even more complex