Sortingccf.ee.ntu.edu.tw/~yen/courses/ds19F/chapter-sort.pdf · 2019-12-07 · Straight Radix Sort...

Sorting

(Sorting) Data Structures Fall 2019 1 / 43

Introduction

One of the most common applications in computer science issorting, the process through which data are arranged according totheir values.If data were not ordered in some way, we would spend anincredible amount of time trying to find the correct information.To appreciate this, imagine trying to find someones number in thetelephone book if the names were not sorted in some way!


General Sorting Concepts

Sorts are generally classified as either internal or external.An internal sort is a sort in which all of the data is held in primarymemory during the sorting process.An external sort uses primary memory for the data currentlybeing sorted and secondary storage for any data that will not fit inprimary memory.For example, a file of 20,000 records may be sorted using an arraythat holds only 1000 records.Therefore only 1000 records are in primary memory at any giventime.The other 19,000 records are stored in secondary storage.Comparison-based sorting: determining order by comparing pairsof elements: <, >, =, ...


Sort Order and Stability

Data may be sorted in either ascending or descending order.The sort order identifies the sequence of sorted data, ascending ordescending.If the order of the sort is not specified, it is assumed to beascending.Sort stability is an attribute of a sort indicating that data elementswith equal keys maintain their relative input order in the output.


Sorting algorithms

Comparison-based sortingbubble sort: swap adjacent pairs that are out of orderselection sort: look for the smallest element, move to frontinsertion sort: build an increasingly large sorted front portionmerge sort: recursively divide the array in half and sort itheap sort: place the values into a sorted tree structurequick sort: recursively partition array based on a middle value...

Other specialized sorting algorithms:bucket sort: cluster elements into smaller groups, sort themradix sort: sort integers by last digit, then 2nd to last, then ...


Selection sort

Order a list of values by repeatedly putting the smallest or largestunplaced value into its final position.

# comparisons = Σni=1(n− i) = O(n2)

# data movements = O(n)(Sorting) Data Structures Fall 2019 6 / 43

Bubble sort


# data movements = Σni=1(n− i) = O(n2)


Insertion sort

Visualize the first part of the list is the sorted portion which isseparated by a c conceptual wall from the unsorted portion of the list.


# data movements = Σni=1(i− 1) = O(n2)

For ”almost sorted sequence”, # comparisons/movements = O(n)


Shell sort

Invented by Donald Shell in 1959.1st algorithm to break the quadratic time barrier but few yearslater, a sub quadratic time bound was provenShellsort works by comparing elements that are distant ratherthan adjacent elements in an array.Shellsort uses a sequence h1, h2, ..., ht called the incrementsequence. Any increment sequence is fine as long as h1 = 1 andsome other choices are better than others.Shellsort improves on the efficiency of insertion sort by quicklyshifting values to their destination.


Shell sort example using sequence (1, 2, 4)

Gap=4

Gap=2


Shell sort example using sequence (1, 2, 4)

Gap=1

Shellsort never does more than n1.5 comparisons (for theh = 1, 4, 13, 40, ...).The analysis of this algorithm is hard. Two conjectures of thecomplexity are n(logn)2 and n1.25


Merge sort

Repeatedly divides the data in half, sorts each half, and combines thesorted halves into a sorted whole.

Divide the list into two roughly equal halves.Sort the left half.Sort the right half.Merge the two sorted halves into one sorted list.

- Often implemented recursively.- An example of a ”divide and conquer” algorithm. Invented by

John von Neumann in 1945


Merge sort example


Merging two sorted subarrays


Quicksort

Follows the divide-and-conquer paradigm.Divide: Partition (separate) the array A[p..r] into two (possiblyempty) subarrays A[p..q− 1] and A[q + 1..r].

I Each element in A[p..q− 1] < A[q].I A[q] < each element in A[q + 1..r].I Index q is computed as part of the partitioning procedure.

Conquer: Sort the two subarrays by recursive calls to quicksort.Combine: The subarrays are sorted in place no work is needed tocombine them.How do the divide and combine steps of quicksort compare withthose of merge sort?


Quicksort - example


Quicksort - example (cont’d)


Quicksort - best case and worst case

Best case: O(n log n)

Worst case: O(n2)


Quicksort - average case

T(n) =1n

n∑k=1

(

pivot at position k︷︸︸︷T(k− 1) + T(n− k)) +

partition︷︸︸︷n + 1

T(n) =2n

n−1∑k=0

T(k) + n + 1

nT(n) = 2n−2∑k=0

T(k) + (n + 1)n

(n− 1)T(n− 1) = 2n−1∑k=0

T(k) + n(n− 1)

nT(n)− (n− 1)T(n− 1) = 2T(n− 1) + 2n

nT(n) = (n− 1)T(n− 1) + 2T(n− 1) + 2n = (n + 1)T(n− 1) + 2n


Quicksort - average case

T(n)

n + 1=

T(n− 1)

n+

2n + 1

=T(n− 2)

n− 1+

2n

+2

n + 1

=T(n− 3)

n− 2+

2n− 1

+2n

+2

n + 1...

=T(n− k)

n− k + 1+ 2(

1n− k + 2

+ ... +1n

+1

n + 1)

Note: ∑ 1k≤

∫dkk≤ O(log n)

Hence, T(n) ≤ O(n log n)


Quicksort - implementation

Almost anything you can try to iimprove Quicksort will actuallyslow it downOne good tweak is to switch to a different sorting method whenthe subarrays get small (say, 10 or 12) – to avoid too muchoverhead for small array sizes

Randomized QuicksortSelect the pivot as a random element of the sequence.The expected running time of randomized quick-sort on asequence of size n is O(n log n).

Median of threeCompare just three elements of our (sub)arraythe first, the last,and the middle Take the median (middle value) of these three aspivotMedian of three is a good technique for choosing the pivot.


Time complexity of Sorting

Several sorting algorithms have been discussed and the best ones,so far:

I Heap sort and Merge sort: O(n log n)I Quick sort (best one in practice): O(n log n) on average, O(n2) worst

caseCan we do better than O(nlogn)?

I No.I It can be proven that any comparison-based sorting algorithm will

need to carry out at least O(n log n) operations


Restrictions on the problem

Suppose the values in the list to be sorted can repeat but thevalues have a limit (e.g., values are digits from 0 to 9)Sorting, in this case, appears easierIs it possible to come up with an algorithm better than O(n log n)?

I YesI Strategy will not involve comparisons


Bucket sort

Idea: suppose the values are in the range 0..m− 1; start with mempty buckets numbered 0 to m− 1, scan the list and placeelement s[i] in bucket s[i], and then output the buckets in orderWill need an array of buckets, and the values in the list to besorted will be the indexes to the buckets

Time complexity is O(n + m)(Sorting) Data Structures Fall 2019 24 / 43

Straight Radix Sort

If your integers are in a larger range then do bucket sort on eachdigitStart by sorting with the low-order digit using a STABLE bucketsort.Then, do the next-lowest,and so on

Time complexity = O(bn)


Straight Radix Sort - example


Straight Radix Sort - example (cont’d)


Straight Radix Sort - correctness

We show that any twokeys are in the correctrelative order at the endof the algorithmGiven two keys, let k bethe leftmost bit-positionwhere they differAt step k the two keysare put in the correctrelative order Because ofstability, the successivesteps do not change therelative order of the twokeys


Radix Exchange Sort


Radix Exchange Sort (cont’d)

Time complexity = O(bn)


Radix Exchange Sort vs. Quicksort

SimilaritiesI both partition arrayI both recursively sort sub-arrays

DifferencesI Method of partitioning

F radix exchange divides array based on greater than or less than 2b−1

F quicksort partitions based on greater than or less than some elementof the array

I Time complexityF Radix exchange O(bn)F Quicksort average case O(n log n)


Lower bound on comparison sorting – decision tree

We may describe the behavior of a comparison-based sortingalgorithm S on an input array A = 〈A[1], ...,A[n]〉 by a decision tree:

At each leaf of the tree the output of the algorithm on thecorresponding execution branch will be displayed. Outputs of sortingalgorithms correspond to permutations of the input array.


Decision tree - example

Insertion sort for n = 3

In insertion sort, when we get the result of a comparison, we oftenswap some elements of the array. In showing decision trees, we dontimplement a swap. Our indices always refer to the original elements atthat position in the array. To understand what I mean, draw theevolving array of Insertion Sort beside this decision tree.


Size of Decision Tree

When dealing with n elements we have n! possible arrangements andneed a decision tree with at least blog n!c levels.


Characteristic Diagrams


Selection Sorting a Random Permutation


Insertion Sorting a Random Permutation


Shell Sorting a Random Permutation


Merge Sorting a Random Permutation


Straight Radix Sort


Quicksort


Heapsorting a Random Permutation: Construction


Heapsorting (Sorting Phase)


Bubble Sorting a Random Permutation


Sortingccf.ee.ntu.edu.tw/~yen/courses/ds19F/chapter-sort.pdf · 2019-12-07 · Straight Radix Sort...

Documents

Transcript of Sortingccf.ee.ntu.edu.tw/~yen/courses/ds19F/chapter-sort.pdf · 2019-12-07 · Straight Radix Sort...