Merge Quick Sort

Design and Analysis of Algorithms – Chapter 4 1

Divide and Conquer (I)

Dr. Ying [email protected]

CSCE 310: Data Structures & CSCE 310: Data Structures & AlgorithmsAlgorithms


Giving credit where credit is due:Giving credit where credit is due:• Most of the lecture notes are based on the slides from Most of the lecture notes are based on the slides from

the Textbook’s companion websitethe Textbook’s companion website– http://www.aw-bc.com/info/levitinhttp://www.aw-bc.com/info/levitin

• Some examples and slides are based on lecture notes Some examples and slides are based on lecture notes created by Dr. Ben Choi, Louisiana Technical University created by Dr. Ben Choi, Louisiana Technical University and Dr. Chuck Cusack, Hope College and Dr. Chuck Cusack, Hope College

• I have modified many of their slides and added new I have modified many of their slides and added new slides.slides.

CSCE 310: Data Structures & CSCE 310: Data Structures & AlgorithmsAlgorithms


Divide and ConquerDivide and Conquer

The most well known algorithm design strategy:The most well known algorithm design strategy:

1.1. Divide instance of problem into two or more smaller Divide instance of problem into two or more smaller instancesinstances

1.1. Solve smaller instances recursivelySolve smaller instances recursively

1.1. Obtain solution to original (larger) instance by combining Obtain solution to original (larger) instance by combining these solutionsthese solutions


Divide-and-conquer TechniqueDivide-and-conquer Technique

subproblem 2 of size n/2

subproblem 1 of size n/2

a solution to subproblem 1

a solution tothe original problem

a solution to subproblem 2

a problem of size n


Divide and Conquer ExamplesDivide and Conquer Examples

Sorting: mergesort and quicksortSorting: mergesort and quicksort

Tree traversalsTree traversals

Binary searchBinary search

Matrix multiplication-Strassen’s algorithmMatrix multiplication-Strassen’s algorithm

Convex hull-QuickHull algorithmConvex hull-QuickHull algorithm


MergesortMergesort

Algorithm:Algorithm: Split array A[1..Split array A[1..nn] in two and make copies of each half in ] in two and make copies of each half in

arrays B[1.. arrays B[1.. nn/2 ] and C[1.. /2 ] and C[1.. nn/2 ]/2 ]

Sort arrays B and CSort arrays B and C Merge sorted arrays B and C into array AMerge sorted arrays B and C into array A


Using Divide and Conquer: Using Divide and Conquer: MergesortMergesort

Mergesort StrategyMergesort Strategy

Sorted

Merge

Sorted Sorted

Sort recursively by Mergesort

Sort recursively by Mergesort

first last

(first last)2


MergesortMergesort



Sort arrays B and CSort arrays B and C Merge sorted arrays B and C into array AMerge sorted arrays B and C into array A


MergesortMergesort



Sort arrays B and CSort arrays B and C Merge sorted arrays B and C into array A as follows:Merge sorted arrays B and C into array A as follows:

• Repeat the following until no elements remain in one of the arrays:Repeat the following until no elements remain in one of the arrays:– compare the first elements in the remaining unprocessed portions of compare the first elements in the remaining unprocessed portions of

the arraysthe arrays

– copy the smaller of the two into A, while incrementing the index copy the smaller of the two into A, while incrementing the index indicating the unprocessed portion of that array indicating the unprocessed portion of that array

• Once all elements in one of the arrays are processed, copy the Once all elements in one of the arrays are processed, copy the remaining unprocessed elements from the other array into A.remaining unprocessed elements from the other array into A.


Algorithm: MergesortAlgorithm: Mergesort

Input:Input: Array E and indices first and last, such that the Array E and indices first and last, such that the elements E[i] are defined for first <= i <= last.elements E[i] are defined for first <= i <= last.Output:Output: E[first], …, E[last] is a sorted rearrangement of E[first], …, E[last] is a sorted rearrangement of the same elementsthe same elementsvoid mergeSort(Element[] E, int first, int last)void mergeSort(Element[] E, int first, int last)if (first < last)if (first < last)int mid = int mid = (first+last)/2(first+last)/2;;mergeSort(E, first, mid);mergeSort(E, first, mid);mergeSort(E, mid+1, last);mergeSort(E, mid+1, last);merge(E, first, mid, last);merge(E, first, mid, last);

return;return;


In-Class ExerciseIn-Class Exercise

P128 Problem 6: Apply mergesort to sort the list E, X, A, P128 Problem 6: Apply mergesort to sort the list E, X, A, M, P, L, E in alphabetical order. M, P, L, E in alphabetical order.


Evaluating Sort AlgorithmsEvaluating Sort Algorithms

Run-timeRun-time: The number of basic operations performed (e.g., : The number of basic operations performed (e.g., compare and swap)compare and swap)

MemoryMemory: The amount of memory used beyond what is : The amount of memory used beyond what is needed to store the data being sortedneeded to store the data being sorted• ““In place” algorithms use a constant amount of extra memory—the In place” algorithms use a constant amount of extra memory—the

constant may be zeroconstant may be zero

• Other algorithms are described as linear or exponential with Other algorithms are described as linear or exponential with respect to the space used.respect to the space used.

• Less is better, but there is often a space/time trade-off.Less is better, but there is often a space/time trade-off.

StabilityStability: An algorithm is stable if it preserves the relative : An algorithm is stable if it preserves the relative order of equal keysorder of equal keys


Algorithm: MergesortAlgorithm: Mergesort

void mergeSort(Element[] E, int first, int last)void mergeSort(Element[] E, int first, int last)

if (first < last)if (first < last)int mid = int mid = (first+last)/2(first+last)/2;;mergeSort(E, first, mid);mergeSort(E, first, mid);

mergeSort(E, mid+1, last);mergeSort(E, mid+1, last);

merge(E, first, mid, last);merge(E, first, mid, last);

return;return;


Mergesort complexityMergesort complexity

Mergesort always partitions the array equally.Mergesort always partitions the array equally. Thus, the recursive depth is always Thus, the recursive depth is always (lg (lg nn) ) The amount of work done at each level is The amount of work done at each level is ((nn) ) Intuitively, the complexity should be Intuitively, the complexity should be ((nn lg lg nn) )

We have, We have, • TT((nn) =2) =2TT((nn/2) /2) ((n) for n>1, T(1)=0 n) for n>1, T(1)=0 ((nn lg lg nn))

The amount of extra memory used is The amount of extra memory used is ((nn)) Note: Mergesort is stableNote: Mergesort is stable


Efficiency of mergesortEfficiency of mergesort

Number of comparisons is close to theoretical minimum for Number of comparisons is close to theoretical minimum for comparison-based sorting: comparison-based sorting: • lg lg n n ! ≈ ! ≈ nn lg lg n n - 1.44- 1.44 n n

Space requirement: Space requirement: ΘΘ( ( n n ) () (NOTNOT in-place) in-place)

Can be implemented without recursion (bottom-up)Can be implemented without recursion (bottom-up)

All cases have same efficiency: All cases have same efficiency: ΘΘ( ( n n log log nn) )


AnimationAnimation

http://math.hws.edu/TMCM/java/xSortLab/index.html


The master theoremThe master theorem

Given: a Given: a divide and conquerdivide and conquer algorithm algorithm

Then, the Master Theorem gives us a cookbook for the Then, the Master Theorem gives us a cookbook for the algorithm’s running time:algorithm’s running time:


A general divide-and-conquer A general divide-and-conquer recurrencerecurrence

TT((nn) = ) = aTaT((n/bn/b) + ) + f f ((nn)) where where f f ((nn)) ∈∈ ΘΘ((nndd))

T(1) = cT(1) = c

a < ba < bdd T T((nn) ) ∈∈ ΘΘ((nndd)) a = ba = bdd T T((nn) ) ∈∈ ΘΘ((nnd d lg lg n n )) a > ba > bdd T T((nn) ) ∈∈ ΘΘ((nnlog log b b aa))

Please refer to Appendix B (P483) for the proofPlease refer to Appendix B (P483) for the proof


In-class exerciseIn-class exercise

Page 128: Problem 5Page 128: Problem 5 5. Find the order of growth for solutions of the following 5. Find the order of growth for solutions of the following

recurrences.recurrences.• a. T(n) = 4T(n/2) + n, T(1) = 1a. T(n) = 4T(n/2) + n, T(1) = 1

• b. T(n) = 4T(n/2) + nb. T(n) = 4T(n/2) + n2222, T(1) = 1, T(1) = 1

• c. T(n) = 4T(n/2) + nc. T(n) = 4T(n/2) + n3333, T(1) = 1, T(1) = 1

In-Class ExerciseIn-Class Exercise

4.1.9 Let A[0, n-1] be an array of n distinct real 4.1.9 Let A[0, n-1] be an array of n distinct real numbers. A pair (A[i], A[j]) is said to be an numbers. A pair (A[i], A[j]) is said to be an inversion if these numbers are out of order, i.e., i<j inversion if these numbers are out of order, i.e., i<j but A[i]>A[j]. Design an O(nlogn) algorithm for but A[i]>A[j]. Design an O(nlogn) algorithm for counting the number of inversions. counting the number of inversions.



Quicksort by Hoare (1962)Quicksort by Hoare (1962)

Select a Select a pivotpivot (partitioning element) (partitioning element) Rearrange the list so that all the elements in the positions Rearrange the list so that all the elements in the positions

before the pivot are smaller than or equal to the pivot and before the pivot are smaller than or equal to the pivot and those after the pivot are larger than or equal to the pivot those after the pivot are larger than or equal to the pivot

Exchange the pivot with the last element in the first (i.e., ≤) Exchange the pivot with the last element in the first (i.e., ≤) sublist – the pivot is now in its final positionsublist – the pivot is now in its final position

Sort the two sublists recursivelySort the two sublists recursively

p

A[i]≤p A[i]p


The partition algorithmThe partition algorithm


The partition algorithmThe partition algorithm

A “sentinel” at A[n] to prevent i advances beyond position n. A “sentinel” at A[n] to prevent i advances beyond position n.


Quicksort ExampleQuicksort Example

Recursive implementation with the left most array Recursive implementation with the left most array entry selected as the pivot element.entry selected as the pivot element.


QuicksortQuicksort

Animated Example: Animated Example: http://math.hws.edu/TMCM/java/xSortLab/index.html


Quicksort AlgorithmQuicksort Algorithm

Input: Array E and indices first, and last, s.t. elements E[i] Input: Array E and indices first, and last, s.t. elements E[i] are defined for first are defined for first i i last last

Ouput: E[first], …, E[last] is a sorted rearrangement of the Ouput: E[first], …, E[last] is a sorted rearrangement of the arrayarray

Void quickSort(Element[] E, int first, int last)Void quickSort(Element[] E, int first, int last)if (first < last)if (first < last)Element pivotElement = E[first];Element pivotElement = E[first];int splitPoint = partition(E, pivotElement, first, last);int splitPoint = partition(E, pivotElement, first, last);quickSort (E, first, splitPoint –1 );quickSort (E, first, splitPoint –1 );quickSort (E, splitPoint +1, last );quickSort (E, splitPoint +1, last );return;return;


Quicksort AnalysisQuicksort Analysis

Partition can be done in O(Partition can be done in O(nn) time, where ) time, where nn is is the size of the arraythe size of the array

Let Let TT((nn) be the number of compares required ) be the number of compares required by Quicksortby Quicksort

If the pivot ends up at position If the pivot ends up at position kk, then we have, then we have• TT((nn) ) TT((nnkk) ) TT((kk 1) 1) nn

To determine best-, worst-, and average-case To determine best-, worst-, and average-case complexity we need to determine the values of complexity we need to determine the values of kk that correspond to these cases.that correspond to these cases.


Best-Case ComplexityBest-Case Complexity

The best case is clearly when the pivot always The best case is clearly when the pivot always partitions the array equally.partitions the array equally.

Intuitively, this would lead to a recursive depth of Intuitively, this would lead to a recursive depth of at most lg at most lg nn calls calls

We can actually prove this. In this case We can actually prove this. In this case • TT((nn) ) TT((nn/2) /2) TT((nn/2) /2) n n ((nn lg lg nn))


Worst-Case and Average-Case Worst-Case and Average-Case ComplexityComplexity

The worst-case is when the pivot always ends up in The worst-case is when the pivot always ends up in the first or last element. That is, partitions the the first or last element. That is, partitions the array as unequally as possible.array as unequally as possible.

In this case In this case • TT((nn) ) TT((nn11) ) TT(1(11) 1) n n TT((nn11) ) nn

nn ( (nn11) ) … + 1 … + 1

nn((nn 1)/2 1)/2 ((nn22))

Average case is rather complex, but is where the Average case is rather complex, but is where the algorithm earns its name. The bottom line is:algorithm earns its name. The bottom line is:

)lg(lg386.1)( nnnnnA


Summary of quicksortSummary of quicksort

Best caseBest case: split in the middle — : split in the middle — ΘΘ( ( n n log log nn) ) Worst caseWorst case: sorted array! — : sorted array! — ΘΘ( ( nn22) ) Average caseAverage case: random arrays — : random arrays — ΘΘ( ( n n log log nn))

Improvements:Improvements:• better pivot selection: median of three partitioning avoids worst better pivot selection: median of three partitioning avoids worst

case in sorted filescase in sorted files• switch to insertion sort on small subfilesswitch to insertion sort on small subfiles• elimination of recursionelimination of recursion

these combine to 20-25% improvementthese combine to 20-25% improvement

Considered the method of choice for internal sorting for Considered the method of choice for internal sorting for large files (large files (nn ≥ 10000) ≥ 10000)

In-class exercisesIn-class exercises

4.2.1 Apply quicksort to sort the list E, X, A, 4.2.1 Apply quicksort to sort the list E, X, A, M, P, L, E in alphabetical order. M, P, L, E in alphabetical order.

Apply quicksort to sort the list 7 2 9 10 5 4Apply quicksort to sort the list 7 2 9 10 5 4

4.2.8 Design an algorithm to rearrange 4.2.8 Design an algorithm to rearrange elements of a given array of n real numbers elements of a given array of n real numbers so that all its negative elements proceed all so that all its negative elements proceed all its positive elements. Your algorithm should its positive elements. Your algorithm should be both time- and space-efficient. be both time- and space-efficient.


Merge Quick Sort

Documents

Transcript of Merge Quick Sort