4.Divide and Conquer

Analysis and Design of Algorithms Chapter 4 – Divide And Conquer

Divide-and-Conquer

Chapter 4

Objective

Divide-and-Conquer is probably well known algorithm design technique. It is well suited for parallel computations, in which one problem can be solved simultaneously by its own processor.

Divide and Conquer

The most well known algorithm design strategy:

1. Divide instance of problem into two or more smaller instances

2. Solve smaller instances recursively

3. Obtain solution to original (larger) instance by combining these solutions

Prepared By Manjunath Kammar


As shown in the figure the case of dividing a problem into two sub problems, by far the most far widely occurring case

These are designed in such away that these can run in single processor.

Appling this method for sum of n natural numbers we can solve this by

Dividing the array of n numbers into two n/2 sub array. Recursively divide the array two equal parts We know that when there is only one number the sum is the

number only. Then add their value to get the sum in equations.

a0+………..+an-1=( a0+…………+a n/2 -1) +( a n/2 +………..+an-1)

In this problem the input of size n can be divided into several instances of size n/b, with a of them needing to be solved. (Here, a and b are constants; a≥1 and b>1).

Assuming that size n is a power of b, to simplify our analysis, we get the following recurrence for the running time t(n):

T(n)= aT(n/b)+f(n).

Where f(n) is the time spent in dividing the problem into smaller ones and then combining their solutions.

T(n) depends on the values of the constants a and b and the order of growth of the function f(n).

Divide and Conquer Examples

Sorting: mergesort and quicksort

Tree traversals

Binary search

Matrix multiplication-Strassen’s algorithm

Master Theorem

Master Theorem

Master Theorem



Master Theorem

• IF f (n) ∈ Θ(nd) where d≥0 in the general divide and conquer recurrence T(n) = aT(n/b) + f (n) with a≥1 and b>1 then,

• Ex: Consider the problem of computing the sum of n numbers recursively.

a0+a1+a2+…. +an-1 = ( a0+…+.an/2 ) + (an/2+…. +an-1 )

A(n) = 2A(n/2)+1

A=2, b=2 and d=0

a > bd

Therefore A(n) ∈ Θ(nlog

b a) = Θ(n)

Mergesort

Merge sort is a perfect example of a successful application of the divide-and-conquer technique.

It sorts a given array A [0….n-1] by dividing it into two halves A[0...

n/2 -1] and A[ n/2….n-1].

Sort each of them recursively.

Merging the smaller sorted arrays into a single sorted one.



Merge

This can be done as follows:

Two pointers (array indices) are initialized to point to the first elements of the arrays being merged.

Then the value of the index compared and smaller of them is added to new array being constructed.

Now the index of the smaller element array is incremented to point to its immediate successor in the array it was copied from.

This operation is continued until one of the two given arrays is exhausted.

The elements of the remaining array will be copied to the end of the new array.



Merge sort Example

Efficiency of merge sort



• Assuming n is a power of 2

• For the worst Case

This is because after every key comparission the total element to be processed will be reduced by 1.

As this is the worst case the smallest number may come from the alternating arrays.

So total comparission is n-1.

Cworst (n) = Θ( n log n)

Quick sort (Partition exchange Sort)

It was invented by a British scientist C.A.R Hoare.

Here the divide and conquer technique is used in the manner:

Divide: Divide the array A[0] A[1]……A[n-1] into two Sub arrays.

A[k]

We continue the same procedure for the sub arrays also.

Conquer: Sort the left part of the Array.

Sort the right part of the array.

Then the resulting array will be sorted.

Merge sort : Input array is divided based on position.


A[k+1] A[k+2]….. A[n-1]

A[0] A[1]……. A[k-1]

Pivot Left part Right Part


Quick Sort : Input array is divided based on their value.

Method :

Partition the given array of size l to r using the pivot element say a[s] such that all the elements towards the left of pivot are <= pivot and all the elements towards the right of pivot are >= pivot.

Now we have 2 sub arrays of size l-(s-1) and (s+1)-r

Repeat the process for sub arrays.

Left to right scan : if A[i]<=p i=i+1

It starts with the second element. it skips all those elements which are smaller than the pivot and stops when found the bigger or equal element.

Right to left scan : if A[j]>=p j=j-1

It starts with the last element of the sub array. It skips all those elements which are bigger smaller than the pivot and stops when found the smaller or equal element.

Three cases may arise depending on whether or not the indices are crossed over or not.

if(i<j) i.e. i and j have not crossed.

We simply exchange a[i] and a[j] and continue scanning by incrementing i and decrementing j.



i j

if(i>j) ) i.e. i and j have crossed over.

Then we have partitioned the array after exchanging the pivot with the A[j].

j i

if(i==j) i.e. when the scanning indices stops while pointing to the same element.

Here the value they are pointing to must be equal to p.

i=j

The partition algorithm

ALGORITHM Partition(A[l….r])

// Partitions a sub array by using its first element as a pivot.

// Input : A Sub array A[l….r] of A[0....n-1], defined by its left and right indices l and r (l<r).

// Output : A partition of A[l…r], with the split position returned as this function’s value.

p← A[l]

i ←l ; j ← r+1

while (true)

repeat i ← i+1 until A[i]≥ p

repeat j ← i-1 until A[i]≤ p

if(i<j) then Swap (A[i],A[j])

else


P all are ≤p ≥ p …………….. ≤ p all are ≥ pP all are ≤p ≥ p …………….. ≤ p all are ≥ p

P all are ≤p ≤ p ≥p all are ≥ p

P all are ≤p =p all are ≥ p


Swap (A[i],A[j])

return j

Quicksort Example

Efficiency of quicksort

Best case :

The number of key comparisons in the best case cbest (n) will satisfy the recurrence.

cbest (n)=2 cbest (n/2)+n for n>1

cbest (1)=0

cbest (n)=2 cbest (n/2)+n

2[2C(n/22)+ (n/2)]+n =>22 C(n/22 )+2n[by replacing n by n/2]



C(n/23)+ 22 (n/22) =>23 C(n/23 )+3n [by replacing n by n/2]

2k C(n/2k)+ k.n

2k C(n/n)+k.n [putting n=2k then k= log2

n]

Therefore cbest (n) = n log2 n

Worst case: sorted array — Θ( n2)

This worst case is because of the array in sorted order. In this we will get the one of the partitioned array as empty, and the size of another is one less than the array which is partitioned from.

In this case of the sorted array in ascending order where A [0] is the pivot then the left to right to scan will stop ay A [1].

Right to scan will stop at A [0].

So we make n+1 comparisons to get to this partition and exchanging with the pivot element A[0] with itself.[this is because the array alady sorted].

This sorting arrays of diminishing sizes will continue until the last one A[n-2..n-1]has been processed.

cworst (n)=(n+1)+n+…….3

This is because for n elements n+1 comparisons.

So for 2 elements 3 comparisons. When one element remaining then no need to be sorted.

So

cworst (n)=((n+1)+n+…….3+(2+1)) – 3

(n+1)(n+2)/2 -3

€ Θ(n2)

Average case : random arrays — Θ( n log2 n)

But in the average case sorting needs 38% more comparisons.



The recurrence relation is given by

n-2

cavg (n)=1/n ∑ (n+1)+ cavg (s) + cavg (n-1-s)] for n>1

i=0

cavg (0)=0, cavg (1)=0.

[where s is a partition 0≤ s ≤n-1]

cavg (n)=1.38 (nlog2n)

Binary Search

Defn: A binary search is a simple technique which can be applied if the items to be compared are either in ascending or descending order.

Very efficient algorithm for searching in sorted array

K

A[0] . . . A[0..m-1] A[m] A[m+1]…… A[n-1]

If K = A[m], stop (successful search); otherwise, continue

searching by the same method in A[0..m-1] if K < A[m]

and in A[m+1..n-1] if K > A[m]

To search an element 83

0 1 2 3 4 5 6 7 8

23 35 45 54 58 65 75 83 95


K<A[m] K>A[m]


l m r

l m r

l,mr

Analysis of binary search:

To analyze the efficiency of binary search we count the number of key comparisons made.

In the best case: Cbest(n) = Ω(1).

In the worst case: Cw (n) = 1 + Cw( ën/2û ), Cw (1) = 1

solution: Cw(n) = log 2n + 1

Binary Tree Traversals And Related Properties.

Defn: A binary tree is defined as a set of nodes that is either empty or consists of a root and two disjoint binary trees TL and TR called, respectively the left and right sub tree of the root.


Iteration1

Iteration2

Iteration3


As the binary tree is divided into two parts many problems can be solved using divide-conquer technique.

Height of a tree: It is defined as the length of the longest path from the root to a leaf.

Height of an empty tree is -1.

Height of a tree with one node is 0.

Height of a tree with two nodes is 1.

The recursive algorithm is

Algorithm Height(T)

//computes recursively the height of a binary tree.

//input: A binary tree T

//output: The height of T.

if T ¹ Æ return -1.

else return maxheight(TL), height (TR) + 1

T TL R

A Standard representation of a binary tree.

The number of comparisons made to compute the maximum of two numbers

and the number of additions A(n(T)) made by the algorithm are the me.

The recurrence relation is given for A(n(T))



A(n(T))= A(n/2(TL))+ A(n/2(TR))+1 for n(T)>0

A(n(T))=0 n(T)=0

A(n(T))= 2 A(n/2(T))+1

Applying master theorem,

A=2, b=2, d=0

a>bd i.e. Θ n logb

a

Θ n log2

2

Θ(n)

Tracing:

Number of comparisons is more than number of additions.

When T=Æ we are not performing any addition, we do one comparison

Therefore it is necessary to calculate number of comparisons along with additions.

n Additions Comparisons

n=0 0 1

n=1 1 3

n=2 2 5


7

5 8

6 9

-1 -1

1

-1 -1

0

1

0

-1 -1

0

2

Height=2


Thus we replace the empty sub trees by special nodes. These nodes are called as external.

The original nodes are called as internal.

So by definition we can say that a empty binary tree is a single external node.

Numbers of external nodes are always one greater than the number of internal nodes.

Number of internal nodes gives number of additions.

Sum of internal and external nodes gives the number of comparisons to be made.

Therefore A(n(T))= n

A(c(T))= n+n+1= 2n+1.

Prove that number of external nodes is always one more than the number of internal nodes.

Basis of induction:

When n=1 we know that the number of internal nodes is 1 and number of external nodes are 2.

Therefore the given statement is true for n=1.


7

5 8

6 9

External

Internal


Assume the given statement is true for n=k such that n=k.

X = k + 1

Let T be the extended binary tree with ‘n’ internal nodes and ‘x’ external nodes.

Let nL and xL be the internal and external nodes of the left sub tree of T.

Similarly nR and xR be the internal and external nodes of the right sub tree of T.

Since n>0 the binary tree has a root , which is an internal node.

Number of internal nodes n= nL+ nR +1

Number of external nodes = xL+ xR

(nL+1)+( nR +1)

( nL+ nR +1)+1

n+1

therefore number of additions required is n.

number of comparison required is 2n+1.

Algorithm for preorder , inorder and postorder traversals.

Algorithm preorder(T)

//purpose: this algorithm will traverse the binary tree in node[N],left[L], and right[R] order.

// input: A binary tree.

// output: An preorder traversed list.

if(T=Æ) return

print (info(T)) //visit the node

preorder(TL) //visit the left sub tree TL in preorder


External internal


preorder(TR) // visit the right sub tree TR in preorder

Algorithm inorder(T)

//purpose: this algorithm will traverse the binary tree in left[L],node[N]and right[R] order.


// output: An inorder traversed list.

if(T=Æ) return

inorder(TL) //visit the left sub tree TL in inorder


inorder(TR) // visit the right sub tree TR in inorder

Algorithm postorder(T)

//purpose: this algorithm will traverse the binary tree in left[L], right[R] and node[N] order.


// output: An postrder traversed list.

if(T=Æ) return

postorder(TL) //visit the left sub tree TL in potsorder

postorder(TR) // visit the right sub tree TR in postorder


Multiplication of Large Integers

Ex: 29 * 15 = 435

29 = 2*101 + 9*100 and 15 = 1*101 + 5*100

29 * 15 = (2*101 + 9*100 ) * (1*101 + 5*100)

= (2*1)102 + (9*1 + 2*5 )101 + (9*5)100

= 200 + 190 + 45



= 435

We can reduce 2 multiplication in the middle term with multiplication.

(9*1 + 2*5 ) = (2+9)*(1+5) –(2*1) –(9*5)


We can obtain the following formula for any pair of two digit numbers a=a1a0 & b=b1b0 their product c can be computed by the following formula.

C = a*b = c2102 + c1101 +c0

C2 = a1*b1 product of their first digits

C0 = a0*b0 product of their second digits

C1 = (a1+a0) * (b1+b0) - (c2+c1)


For 2 n-digit integers where n is positive even number.

a=a1a0 b=b1b0

First half of a ga1 First half of b gb1

Second half of a ga0 Second half of b gb0

a=a1a0 b=b1b0

g a=a110n/2 + a0 g b=b110n/2 + b0

C=a*b = (a110n/2 + a0)*( b=b110n/2 + b0)

= (a1*b1)10n +(a1*b0+a0*b1)10n/2+(a0*b0)



= c210n + c110n/2 +c0

Where

C2 = a1*b1 product of their first digits

C0 = a0*b0 product of their second digits

C1 = (a1+a0) * (b1+b0) - (c2+c1)

Analysis:

Multiplication of n-digit numbers requires three multiplications of n/2-digit numbers, the recurrence for the number of multiplications M (n) will be

M (n)=3 M (n/2) for n>1

M (1)=1.

Consider

M (n)=3 M (n/2)

M (n)=32 M (n/22) putting n=n/2

M (n)=3k M (n/2k) substituting 2k=n

M (n)=3k

M (n)=3log2

n

M (n)=n log2

3 ≈ n1.585 <n2

Strassen’s matrix multiplication

m1 = (a00 + a11) * (b00 + b11)

m2 = (a10 + a11) * b00

m3 = a00 * (b01 - b11)

m4 = a11 * (b10 - b00)

m5 = (a00 + a01) * b11



m6 = (a10 - a00) * (b00 + b01)

m7 = (a01 - a11) * (b10 + b11)

Strassen’s matrix multiplication

c00 c01 a00 a01 b00 b01

= *

c10 c11 a10 a11 b10 b11

m1 + m4 - m5 + m7 m3 + m5

=

m2 + m4 m1 + m3 - m2 + m6



Efficiency of Strassen’s algorithm

If M(n) is the number of multiplications made by in multiplying 2 n-by-n matrices,

It is smaller than n3 required by the brute force algorithm.

Since this saving in the number of multiplications was achieved at the expense of making extra additions., we must check the number of additions A(n) made by strassen’s algorithm.

Questions:

1. Write an algorithm to mergesort using divide and conquer strategy. Trace the algorithm for input set 4,7,1,3,8,5. (July-2006)(10 marks)

2. Discuss strassen’s matrix multiplication algorithm. Find its time complexity. (July-2006)(10 marks).

3. Give the algorithm for merge sort and trace it’s operation on the following sequence of numbers. (Dec – 2005) (10 Marks)

9 2 3 7 6 4 8 1



4. Describe strassen’s matrix multiplication algorithm and evaluate the asymptotic efficiency. (Jan-Feb 2005) (10 marks).

5. Write a pseudo code for a merge sort algorithm. Set up and solve a recurrence relation for the number of key comparisons made by your algorithm. (Jan-Feb 2005) (10 marks)

6. Write quick sort algorithm and apply it to sort list E, X, A, M, P, 7. L, E (July – Aug 2004) (10 Marks).

8. Briefly explain a method to multiply 2 large numbers based on the divide and conquer method. Hence compute the 1234 * 2101 using the same. (July – Aug 2004) (10 Marks).

9. Explain Merge sort with its complexities.

10. Explain Quick sort with its complexities.

11. Explain Binary Search with its complexities.

12. Explain Binary Tree Traversals and Related Properties.

13. Explain Multiplication of Large Integers with its complexities.

14. Explain Matrix Multiplication with its complexities


4.Divide and Conquer

Documents

Transcript of 4.Divide and Conquer