Sorting Quick Sort, Shell Sort, Counting Sort, Radix Sort AND Bucket Sort.
Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort...
Transcript of Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort...
![Page 1: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/1.jpg)
Sorting in Linear Time
Srichand Pendyala
![Page 2: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/2.jpg)
The next few minutes...
1. Lower Bounds for Sorting2. Comparison based sorting3. Counting Sort4. Radix Sort5. Bucket Sort
![Page 3: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/3.jpg)
Comparison Sorting
Comparison sorting uses comparisons of the following forms to decide the relative
order of two elements in a list
a < ba == ba > b
![Page 4: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/4.jpg)
Decision Tree
1:2
2:3 1:3
<1,2,3> 1:3 2:3<2,1,3>
<1,3,2> <3,1,2> <2,3,1> <3,2,1>
![Page 5: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/5.jpg)
More precisely, it can be proved, that the lower bound on comparison based sorting algorithms is Ω(n log n) comparisons in the
worst case.
Lower Bound on Comparison sorting
![Page 6: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/6.jpg)
You want the proof? You can't han...
Determine the lower bound on the heights of decision trees in which each permutation
appears as a reachable leaf.
n! <= l <= 2h h >= lg(n!)
lg(n!) = Ω(n lg n)
![Page 7: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/7.jpg)
Linear Ordering Sort
Linear ordering? Didn't we just prove that sorting takes Ω(n log n) ?
Remember: That's true for comparison based sorting algorithms * only *!
![Page 8: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/8.jpg)
Linear sorts in general
In general linear sorting algorithms use operations other than comparison to
determine the sorted order of the elements.
![Page 9: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/9.jpg)
Quotation
“The first person to sleep during my presentation would be ... me”
- Srichand Pendyala, 2003.
To stay awake: I've got 3 pictures, see if you can connect them! And no, it doesn't
have anything to do with sorting.
![Page 10: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/10.jpg)
Counting Sort
● Counting sort assumes that each of the n input elements in an integer in the range 0 to k.
● Determine, for each input element x, the number of elements less than x.
● So, if there are 41 elements less than x then x is at the 42nd position.
![Page 11: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/11.jpg)
Counting SortCounting-Sort(A,B,k)
for i ←0 to k do C[i] ← 0
for j ← 1 to length[A]do C[A[j]] ← C[A[j]] + 1
for i ← to kdo C[i] ← C[i] + C[i-1]
for j ← length[A] downto 1do B[C[A[j]]] ← A[j]
C[A[j]] ← C[A[j]] -1
# Initialize C to 0s
# C now contains number of elements equal to i
# C now contains the number of elements less than or equal to i
# Rearrange into output array
![Page 12: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/12.jpg)
Counting Sort
An example
Initial array: [5, 1, 4, 6, 2]C = [1, 1, 0, 1, 1, 1]
Making it cumulativeC = [1, 2, 2, 3, 4, 5]
B = [N 2 N N N], C = [1 1 2 3 4 5] B = [N 2 N N 6], C = [1 1 2 3 4 4]B = [N 2 4 N 6], C = [1 1 2 2 4 4]B = [1 2 4 N 6], C = [0 1 2 2 4 4]B = [1 2 4 5 6], C = [0 1 2 2 3 4]
![Page 13: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/13.jpg)
More on Counting sort
Counting sort is stable, ie numbers with the same value appear in the same order in the
output array.
Counting sort's stability is the basis for Radix sort's correctness.
![Page 14: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/14.jpg)
Radix SortRadix sort is (was?) used by card-sorting
machines.
Sorting is based on a counting sort through each digit of the numbers.
Numbers are placed into one of 10 bins, for each of the 10 possible values of each digit
Radix sort sorts the least significant digits first. Sorting by most significant digits
requires recursive sorting in addition to keeping 9/10ths of the numbers aside.
![Page 15: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/15.jpg)
Radix Sort
Radix-Sort(A, d)
for i ← 1 to ddo use a stable sort to sort array A on digit i
![Page 16: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/16.jpg)
Radix Sort
Example:
A = [101, 512, 123, 521, 23, 989]A = [101, 521, 512, 123, 23, 989]A = [101, 512, 521, 123, 23, 989]A = [23, 101, 123, 512, 521, 989]
![Page 17: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/17.jpg)
Radix Sort
How does Radix Sort compare to, say, Merge Sort?
Radix sort, which uses counting sort as the intermediate stable sort, does not sort in
place.
![Page 18: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/18.jpg)
Bucket Sort
Bucket sort is able to sort in linear time, because it assumes something about its input, just like counting sort. It assumes
that the input array is uniformly distributed in the range [0,1)
Not stable!
It divides the interval [0,1) into n equal sized subintervals, or buckets and then distribute the n input numbers into the
buckets.
![Page 19: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/19.jpg)
Bucket Sort
Bucket-Sort(A)n ← length[A]for i ← 1 to n
do insert A[i] into list B[[nA[i]]
for i ← 0 to n-1do sort list B[i] with insertion sort
concatenate the lists B[0], B[1], ..., B[n-1] together in order
![Page 20: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/20.jpg)
Bucket Sort
.78
.17
.39
.26
.72
.94
.21
.12
.23
.68
.21 -> .23 -> .26
.39
/
/
.68
.72 -> .78
/
.12 -> .17
/
.94
![Page 21: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/21.jpg)
Summary
• Comparison based sorting algorithms, and why they sort of suck.
• Linear ordering and how its sort of possible.
• Counting sort
• Radix Sort
• Bucket Sort
![Page 22: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/22.jpg)
Questions?
“I do not promise an answer to your question. I only merely offer you the
hypothesis that I may have understood it.”- Anonymous, (Not sure when)
(Did he mean it the question, or it the answer?)
![Page 23: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/23.jpg)
The pictures?
Any ideas?
![Page 24: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/24.jpg)
The pictures?
Any ideas?
Tupper's Self Referential Formula
![Page 25: Sorting in Linear Time - pages.mtu.eduaebnenas/teaching/fall2007/cs5321/lectures/… · Radix Sort Radix sort is (was?) used by card-sorting machines. Sorting is based on a counting](https://reader035.fdocuments.in/reader035/viewer/2022062921/5f0421a47e708231d40c77fe/html5/thumbnails/25.jpg)
Thank you!
“A captive audience is just as good as a captivated one.” - Srichand Pendyala, 2006