Sorting Sorting AlgorithmsAlgorithms

22

QuicksortQuicksort

General Quicksort Algorithm:General Quicksort Algorithm: Select an element from the array to be Select an element from the array to be

the pivotthe pivot Rearrange the elements of the array into Rearrange the elements of the array into

a left and right subarraya left and right subarray All values in the left subarray are < pivotAll values in the left subarray are < pivot All values in the right subarray are > pivotAll values in the right subarray are > pivot

Independently sort the subarraysIndependently sort the subarrays No merging required, as left and right No merging required, as left and right

are independent problems are independent problems [ Parallelism?!? ][ Parallelism?!? ]

QuicksortQuicksort

void quicksort(int* arrayOfInts, int first, int void quicksort(int* arrayOfInts, int first, int last) last)

{ { int pivot; int pivot; if (first < last) if (first < last) { { pivot = partition(arrayOfInts, first, last); pivot = partition(arrayOfInts, first, last); quicksort(arrayOfInts,first,pivot-1); quicksort(arrayOfInts,first,pivot-1); quicksort(arrayOfInts,pivot+1,last); quicksort(arrayOfInts,pivot+1,last); } } } }

QuicksortQuicksortint partition(int* arrayOfInts, int first, int last) int partition(int* arrayOfInts, int first, int last) { { int temp;int temp; int p = first; int p = first; // set pivot = first // set pivot = first

indexindex for (int k = first+1; k <= last; k++) // for every other indxfor (int k = first+1; k <= last; k++) // for every other indx { { if (arrayOfInts[k] <= arrayOfInts[first]) // if data is if (arrayOfInts[k] <= arrayOfInts[first]) // if data is

smallersmaller { {

p = p + 1; p = p + 1; // update final pivot // update final pivot locationlocation swap(arrayOfInts[k], arrayOfInts[p]);swap(arrayOfInts[k], arrayOfInts[p]);

}} } } swap(arrayOfInts[p], arrayOfInts[first]);swap(arrayOfInts[p], arrayOfInts[first]); return p; return p; }}

99 55 33 2020181899 1818 33 20205599 33 1818 202055

Partition Step ThroughPartition Step Through

partition(cards, 0, 4)P = 0 K = 1 P = 1 K = 3

cards[1] < cards[0] ? No cards[3] < cards[0]? YesP = 2

P = 0 K = 2 temp = cards[3]cards[2] < cards[0] ? Yes cards[3] = cards[2]P = 1 cards[2] = cards[3]temp = cards[2] P = 2 K = 4cards[2] = cards[1] cards[4] < cards[0]? Nocards[1] = temp

temp = cards[2], cards[2] = cards[first]cards[first] = temp, return p = 2;

33 99 1818 202055

Complexity of QuicksortComplexity of Quicksort

Worst case is O(nWorst case is O(n22)) What does worst case correspond to?What does worst case correspond to?

Already sorted or near sortedAlready sorted or near sorted Partitioning leaves heavily unbalanced Partitioning leaves heavily unbalanced

subarrayssubarrays

On average is O(n logOn average is O(n log22n), and it is n), and it is average a lot of the time.average a lot of the time.

Complexity of QuicksortComplexity of Quicksort

Recurrence Relation: [Average Case]Recurrence Relation: [Average Case]

2 sub problems2 sub problems

½ size (if good pivot)½ size (if good pivot)

Partition is O(n)Partition is O(n)

a = 2 b = 2 k = 1a = 2 b = 2 k = 1

2 = 22 = 211 Master Theorem: Master Theorem: O(nlogO(nlog22n)n)

Complexity of QuicksortComplexity of Quicksort

Recurrence Relation: [Worst Case]Recurrence Relation: [Worst Case] Partition separates into (n-1) and (1)Partition separates into (n-1) and (1) Can’t use master theorem:Can’t use master theorem:

b (subproblem size) changesb (subproblem size) changes

n-1/n n-2/n-1 n-3/n-2n-1/n n-2/n-1 n-3/n-2 Note that sum of partition work:Note that sum of partition work:

n + (n-1) + (n-2) + (n-3) …n + (n-1) + (n-2) + (n-3) …

Sum(1,N) Sum(1,N) = N(N+1)/2= N(N+1)/2 = = O(NO(N22))

Complexity of QuicksortComplexity of Quicksort

Requires stack space to implement Requires stack space to implement recursionrecursion

Worst case: O(n) stack space Worst case: O(n) stack space If pivot breaks into 1 element and n-1 If pivot breaks into 1 element and n-1

element subarrayselement subarrays Average case: O(log n)Average case: O(log n)

Pivot splits evenlyPivot splits evenly

MergeSortMergeSort General Mergesort Algorithm:General Mergesort Algorithm:

Recursively split subarrays in halfRecursively split subarrays in half Merge sorted subarraysMerge sorted subarrays

Splitting is first in recursive call, so continues Splitting is first in recursive call, so continues until have one item subarraysuntil have one item subarrays

One item subarrays are by definition sortedOne item subarrays are by definition sorted

Merge recombines subarrays so result is Merge recombines subarrays so result is sortedsorted

1+1 item subarrays => 2 item subarrays1+1 item subarrays => 2 item subarrays 2+2 item subarrays => 4 item subarrays2+2 item subarrays => 4 item subarrays Use fact that subarrays are sorted to simplify merge Use fact that subarrays are sorted to simplify merge

algorithmalgorithm

MergeSortMergeSort

void mergesort(int* array, int* tempArray, int low, void mergesort(int* array, int* tempArray, int low, int high, int size)int high, int size)

{{if (low < high)if (low < high)

{{ int middle = (low + high) / 2;int middle = (low + high) / 2; mergesort(array,tempArray,low,middle, size);mergesort(array,tempArray,low,middle, size); mergesort(array,tempArray,middle+1, high, mergesort(array,tempArray,middle+1, high,

size);size); merge(array,tempArray,low,middle,high, size);merge(array,tempArray,low,middle,high, size); }}}}

MergeSortMergeSortvoid merge(int* array, int* tempArray, int low, int middle, int high, int size) void merge(int* array, int* tempArray, int low, int middle, int high, int size) {{ int i, j, k;int i, j, k;

for (i = low; i <= high; i++) { tempArray[i] = array[i]; } // copy into temp for (i = low; i <= high; i++) { tempArray[i] = array[i]; } // copy into temp arrayarray

i = low; j = middle+1; k = low;i = low; j = middle+1; k = low;

while ((i <= middle) && (j <= high)) {while ((i <= middle) && (j <= high)) { // merge// merge if (tempArray[i] <= tempArray[j])if (tempArray[i] <= tempArray[j]) // if lhs item is smaller// if lhs item is smaller

array[k++] = tempArray[i++];array[k++] = tempArray[i++]; // put in final array, increment// put in final array, increment elseelse // final array position, lhs index// final array position, lhs index

array[k++] = tempArray[j++];array[k++] = tempArray[j++]; // else put rhs item in final array// else put rhs item in final array }} // increment final array // increment final array

positionposition// rhs index// rhs index

while (i <= middle)while (i <= middle) // one of the two will run out// one of the two will run out array[k++] = tempArray[i++];array[k++] = tempArray[i++]; // copy the rest of the data// copy the rest of the data}} // only need to copy if in lhs // only need to copy if in lhs

arrayarray // rhs array already in right place// rhs array already in right place

MergeSort ExampleMergeSort Example

Recursively Split

2020 33 1818 99 55

2020 33 1818 99 55

MergeSort ExampleMergeSort Example

Recursively Split

2020 33 1818 99 55

2020 33 181899

55

MergeSort ExampleMergeSort Example

Merge

2020 33 1818 99 55

Merge Sort ExampleMerge Sort Example

Temp Array

i j

Array

Temp[i]< Temp[j]

Yes

2 cardsNot very interestingThink of as swap

2020 33

1818

202033

202033

k

33

MergeSort ExampleMergeSort ExampleTemp Array

i j

Array

Temp[i]< Temp[j]

No

k

Update J, K by 1 => Hit Limit of Internal While Loop, as J > High NowCopy until I > Middle

1818202033 33 1818

k

2020Array

181833

MergeSort ExampleMergeSort Example2 Card Swap

Final aftermergingabove sets

99 55 55 99

2020181833 55 99

33i=1,j=3

55i=1,j=4

i=0,j=3

99i=1,j=5

1818i=2,j=5

2020i=3,j=5

Complexity of MergeSortComplexity of MergeSort

Recurrence relation:Recurrence relation:2 subproblems2 subproblems½ size½ sizeMerging is O(n) for any subproblemMerging is O(n) for any subproblem

Always moving forwards in the arrayAlways moving forwards in the array

a = 2 b = 2 k = 1a = 2 b = 2 k = 12 = 22 = 211 Master Theorem: O(n logMaster Theorem: O(n log22n)n)

Always O(n logAlways O(n log22n) in both average and worst case n) in both average and worst case Doesn’t rely on quality of pivot choiceDoesn’t rely on quality of pivot choice

Space Complexity of Space Complexity of MergesortMergesort

Need an additional O(n) temporary Need an additional O(n) temporary arrayarray

Number of recursive calls:Number of recursive calls: Always O(logAlways O(log22n)n)

TradeoffsTradeoffs

When it is more useful to:When it is more useful to: Just searchJust search Quicksort or Mergesort and searchQuicksort or Mergesort and search

Assume Z searchesAssume Z searchesSearch on random data: Z * O(n)Search on random data: Z * O(n)Fast Sort and binary search: Fast Sort and binary search:

O(nlogO(nlog22n) + Z *logn) + Z *log22nn

TradeoffsTradeoffs

Z * n <= nlogZ * n <= nlog22n + Zlogn + Zlog22nnZ(n - logZ(n - log22n) <= n logn) <= n log22nn

Z <= (n logZ <= (n log22n) / (n-logn) / (n-log22n)n)

Z <= (n logZ <= (n log22n) / n [Approximation]n) / n [Approximation]Z <= logZ <= log22n [Approximation]n [Approximation]

Where as before, had to do N searches to make Where as before, had to do N searches to make up for cost of sorting, now only do logup for cost of sorting, now only do log22NN

1,000,000 items = 19 searches, instead of 1,000,000 items = 19 searches, instead of 1,000,0001,000,000

How Fast?How Fast?

Without specific details of what Without specific details of what sorting, O(n logsorting, O(n log22n) is the maximum n) is the maximum speed sort possible.speed sort possible. Only available operations: Compare, SwapOnly available operations: Compare, Swap

Proof: Decision Tree – describes how Proof: Decision Tree – describes how sort operatessort operates Every vertex represents a comparison, Every vertex represents a comparison,

every branch a resultevery branch a result Moving down tree – Tracing a possible Moving down tree – Tracing a possible

run through the algorithmrun through the algorithm

How Fast?How Fast?

K1 <= K2 [1,2,3]

K2 <= K3 K1 <= K3[1,2,3] [2,1,3]

Yes No

stop K1 <= K3

YesNo

K2 <= K3stop

stop stopstop stop

Yes

Yes

YesNo

No

No

[1,2,3]

[1,3,2]

[2,1,3]

[1,3,2] [3,1,2]

[2,3,1]

[2,3,1] [3,2,1]

How Fast?How Fast? There are n! possible “stop” nodes – There are n! possible “stop” nodes –

effectively all permutations of the n effectively all permutations of the n numbers in the array.numbers in the array.

Thus any decision tree representing a Thus any decision tree representing a sorting algorithm must have n! leavessorting algorithm must have n! leaves

The height of a this type of tree (The height of a this type of tree (a binary a binary treetree) is correlated with number of leaves:) is correlated with number of leaves: Height k = 2^(k-1) leavesHeight k = 2^(k-1) leaves Must be at least logMust be at least log22n! + 1 heightn! + 1 height

How Fast?How Fast? Path from top to bottom of tree – trace of a Path from top to bottom of tree – trace of a

run of the algorithmrun of the algorithm Need to prove that (logNeed to prove that (log22n!) is lower bounded n!) is lower bounded

by (n logby (n log22n)n)

n! = (n)(n-1)(n-2)(n-3) … (3)(2)(1)n! = (n)(n-1)(n-2)(n-3) … (3)(2)(1) > (n)(n-1)(n-2)(n-3) … ceil(n/2) // doing fewer multiplies> (n)(n-1)(n-2)(n-3) … ceil(n/2) // doing fewer multiplies > ceil(n/2) > ceil(n/2) (ciel(n/2))(ciel(n/2)) // doing multiplies of bigger things // doing multiplies of bigger things > approximately (n/2)> approximately (n/2)(n/2)(n/2)

log log 22 n! > log n! > log 2 2 (n/2)(n/2)(n/2) (n/2)

log log 22 n! > (n/2) log n! > (n/2) log 2 2 (n/2)(n/2) //exponentiation in logs = multiplication out //exponentiation in logs = multiplication out frontfront

log log 22 n! > (n/2)(log n! > (n/2)(log22n – logn – log22 2) // division in logs = subtraction 2) // division in logs = subtractionlog log 22 n! > (n/2)(log n! > (n/2)(log22n – 1)n – 1)log log 22 n! > (n/2)(log n! > (n/2)(log22n) – (n/2)n) – (n/2)log log 22 n! > (1/2) [nlog n! > (1/2) [nlog22n – n]n – n]log log 22 n! ~ O(n log n! ~ O(n log22n)n)