Counting sort(Non Comparison Sort)

44
Sorting Algorithms

description

The Algorithm of Counting Sort

Transcript of Counting sort(Non Comparison Sort)

Page 1: Counting sort(Non Comparison Sort)

Sorting Algorithms

Page 2: Counting sort(Non Comparison Sort)

2

Comparison Sorting A comparison sort is a type of sorting algorithm that only

reads the list elements through a single comparison operation (often a "less than or equal to" operator or a three-way comparison) and determines which of two elements should occur first in the final sorted list.

What is common to all these algorithms?

– Make comparisons between input elements like:

ai < aj, ai ≤ aj, ai = aj, ai ≥ aj, or ai > aj

Page 3: Counting sort(Non Comparison Sort)

3

Examples of Comparison Sorting1. Bubble sort

2. Insertion sort

3. Selection sort

4. Quick sort

5. Heap sort

6. Merge sort

7. Odd-even sort

8. Cocktail sort

9. Cycle sort

10. Merge insertion (Ford-Johnson) sort

11. Smoothsort

12. Timsort

Page 4: Counting sort(Non Comparison Sort)

4

Limitations of Comparison Sorting

There are fundamental limits on the performance of comparison sorts.

To sort n elements, comparison sorts must make (n log n) comparisons in the worst case.

That is a comparison sort must have a lower bound of (Ω n log n) comparison operations, which is known as linear

or linearithmic time. This is a consequence of the limited information available

through comparisons alone

Page 5: Counting sort(Non Comparison Sort)

Why does it takes so much of time? A decision tree is used to represent the comparisons of a

sorting algorithm. Assume that all inputs are distinct. A decision tree compares all possible inputs to each other to determine the sequence of outputs.

Decision Tree for three numbers a1, a2, a3 : If at the root, a1 ≤ a2 g ο left and compare a2 to a3, otherwise go to right and compare a1 to a3. Each path represents a different ordering of a1, a2, a3.

Page 6: Counting sort(Non Comparison Sort)

Why does it takes so much of time? This type of decision tree will have n! leaves. Any comparison based sorting algorithm will have to go

through the steps in the decision tree as a minimum

Page 7: Counting sort(Non Comparison Sort)

Why does it takes so much of time? So for n inputs, the tree must have n! leaves. A binary tree of

height h has no more than 2h leaves. Now n ! <= 2h

Taking log on both side: lg(n !) <= h According to stirling’s approximation we have:

So it can be written h= (Ω n log n). This means we need to do at least n log n comparisons to reach the bottom of the tree

Page 8: Counting sort(Non Comparison Sort)

8

How Fast Can We Sort?

• Selection Sort, Bubble Sort, Insertion Sort, Cocktail sort,

Cycle sort: O(n2)

• Smooth sort, Odd-even sort, Timsort: O(n)

• Heap Sort, Merge sort: O(n log n)

• Quicksort:

– Average: O(n log n)

– Best: O(n log n) (simple partition) or O(n) (three-way

partition and equal keys)

Page 9: Counting sort(Non Comparison Sort)

9

Non Comparison Sorting There are some sorting algorithms that perform sorting without

comparing the elements rather by making certain assumptions about the data. They are called the non comparison sorting.

Non comparison sorting include:

1. Counting sort (indexes using key values)

2. Radix sort (examines individual bits of keys)

3. Bucket sort (examines bits of keys) These are Linear sorting algorithms. Linear sorts are NOT

“comparison sorts”. They make certain assumptions about the data. These type of

sorting algorithm does not need to go through the comparison decision tree.

Page 10: Counting sort(Non Comparison Sort)

Counting sort Counting sort assumes that each of the n input elements is an

integer in the range 0 to k. that is n is the number of elements and k is the highest value element.

Consider the input set : 4, 1, 3, 4, 3. Then n=5 and k=4 Counting sort determines for each input element x, the number of

elements less than x. And it uses this information to place element x directly into its position in the output array. For example if there exits 17 elements less that x then x is placed into the 18th position into the output array.

The algorithm uses three array:

Input Array: A[1..n] store input data where A[j] {1, 2, 3, …, k}

Output Array: B[1..n] finally store the sorted data

Temporary Array: C[1..k] store data temporarily

Page 11: Counting sort(Non Comparison Sort)

Counting Sort1. Counting-Sort(A, B, k)

2. Let C[0…..k] be a new array

3. for i=0 to k

4. C[i]= 0;

5. for j=1 to A.length or n

6. C[ A[j] ] = C[ A[j] ] + 1;

7. for i=1 to k

8. C[i] = C[i] + C[i-1];

9. for j=n or A.length down to 1

10. B[ C[ A[j] ] ] = A[j];

11. C[ A[j] ] = C[ A[j] ] - 1;

Page 12: Counting sort(Non Comparison Sort)

Counting Sort1. Counting-Sort(A, B, k)

2. Let C[0…..k] be a new array

3. for i=0 to k [Loop 1]4. C[i]= 0;

5. for j=1 to A.length or n [Loop 2]6. C[ A[j] ] = C[ A[j] ] + 1;

7. for i=1 to k [Loop 3]8. C[i] = C[i] + C[i-1];

9. for j=n or A.length down to 1 [Loop 4]10. B[ C[ A[j] ] ] = A[j];

11. C[ A[j] ] = C[ A[j] ] - 1;

Page 13: Counting sort(Non Comparison Sort)

Counting-sort example

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0 3 3

0

5

1 2 3 4 5 6 7 8

Page 14: Counting sort(Non Comparison Sort)

Executing Loop 1

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0 3 3

0 0 0 0 0 0 0 0 0 0

0

0 0

5

1 2 3 4 5 6 7 8

Page 15: Counting sort(Non Comparison Sort)

Executing Loop 2

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0 3 3

0 0 0 0 1 0 0 0 0

0

0 0

5

1 2 3 4 5 6 7 8

Page 16: Counting sort(Non Comparison Sort)

Executing Loop 2

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 3 3 0 0 2 2 3 3 0 0 3 3

0 0 0 0 1 1 0 0 0 0

0

1

5

1 2 3 4 5 6 7 8

Page 17: Counting sort(Non Comparison Sort)

Executing Loop 2

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 0 0 2 2 3 3 0 0 3 3

0 0 0 0 1 1 1 0 0

0

1 1

5

1 2 3 4 5 6 7 8

Page 18: Counting sort(Non Comparison Sort)

Executing Loop 2

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 2 2 3 3 0 0 3 3

1 0 0 1 1 1 1 0 0

0

1 1

5

1 2 3 4 5 6 7 8

Page 19: Counting sort(Non Comparison Sort)

Executing Loop 2

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 3 3 0 0 3 3

1 1 0 0 2 1 1 0 0

0

1 1

5

1 2 3 4 5 6 7 8

Page 20: Counting sort(Non Comparison Sort)

Executing Loop 2

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 0 0 3 3

1 1 0 0 2 2 2 0 0

0

1 1

5

1 2 3 4 5 6 7 8

Page 21: Counting sort(Non Comparison Sort)

Executing Loop 2

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 3 3

2 0 0 2 2 2 2 0 0

0

1 1

5

1 2 3 4 5 6 7 8

Page 22: Counting sort(Non Comparison Sort)

Executing Loop 2

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0 3

2 2 0 0 2 2 3 0 0

0

1 1

5

1 2 3 4 5 6 7 8

Page 23: Counting sort(Non Comparison Sort)

End of Loop 2

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0

2 2 0 0 2 2 0 0

0

1 1

5

1 2 3 4 5 6 7 8

3 3

3 3

Page 24: Counting sort(Non Comparison Sort)

Executing Loop 3

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0

2 0 2 2 0 0 0

1 1 5

1 2 3 4 5 6 7 8

3 3

3 3

C: 2 2 0 0 1 1 3 3

1 2 3 40 5

2 2 2

Page 25: Counting sort(Non Comparison Sort)

Executing Loop 3

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0

2 2 2 2 0 0 0

1 1 5

1 2 3 4 5 6 7 8

3 3

3 3

C: 2 2 4 0 0 1 1 3 3

1 2 3 40 5

2 2

Page 26: Counting sort(Non Comparison Sort)

Executing Loop 3

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0

4 2 2 0 0 0

1 1 5

1 2 3 4 5 6 7 8

3

3 3

C: 2 2 4 4 0 0 1 1 7

1 2 3 40 5

2 2

2 2

Page 27: Counting sort(Non Comparison Sort)

Executing Loop 3

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0

2 2 0 0

1 1 5

1 2 3 4 5 6 7 8

7

3 3

C: 2 2 4 4 7 1 1

1 2 3 40 5

2 2

2 2

4 4

7 7

Page 28: Counting sort(Non Comparison Sort)

Executing Loop 3

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0

2 2

0

1

5

1 2 3 4 5 6 7 8

3 3

C: 2 2 4 4 8

1 2 3 40 5

2 2

2 2

4 4

7 7

7 7 7

7 7

Page 29: Counting sort(Non Comparison Sort)

End of Loop 3

A:

B:

1 2 3 4 5

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0

1 2 3 4 5 6 7 8

3 3

C: 2 2 4 4

1 2 3 40 5

2 2 7 7 7 7 8 8

Page 30: Counting sort(Non Comparison Sort)

Executing Loop 4

A:

B:

1 2 3 4 5

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0

1 2 3 4 5 6 7 8

3

C: 2 2 4 4

1 2 3 40 5

2 2 7 7 7 8 8

Page 31: Counting sort(Non Comparison Sort)

Executing Loop 4

A:

B:

1 2 3 4 5

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0

1 2 3 4 5 6 7 8

C: 2 2 4 4

1 2 3 40 5

2 2 7 7 7 8 8

3

J=8, then A[ j ]=A[8]=3And B[ C[ A[j] ] ]=B[ C[ 3 ] ]=B[ 7]So B[ C[ A[j] ] ] ←A[ j ]=B[7]←3

Page 32: Counting sort(Non Comparison Sort)

A:

B:

1 2 3 4 5

6 7 8

5 5 3 3 0 0 2 2 3 3

1 2 3 4 5 6 7 8

C: 2 4 4

1 2 3 40 5

2 2 7 7 8 8

3 3 0

6 6

3 3

Executing Loop 4

J=8, then A[ j ]=A[8]=3Then C[ A[j] ] = C[ 3 ] =7So C[ A[j] ] = C[ A[j] ] -1=7-1=6

Page 33: Counting sort(Non Comparison Sort)

Executing Loop 4

A:

B:

1 2 3 4 5

6 7 8

5 5 3 3 0 0 2 2

1 2 3 4 5 6 7 8

C: 4 4

1 2 3 40 5

2 2 7 7 8 8

3 3

6

3 3

0 0

1 1

0 0

3

Page 34: Counting sort(Non Comparison Sort)

Executing Loop 4

A:

B:

1 2 3 4 5

6 7 8

5 5 3 3 0 0

1 2 3 4 5 6 7 8

C: 4

1 2 3 40 5

2 2 7 7 8 8

3 3

5 5

3 3

0 0

0 0

2

1 1

3 3

3 3

Page 35: Counting sort(Non Comparison Sort)

Executing Loop 4

A:

B:

1 2 3 4 5

6 7 8

5 5 3 3

1 2 3 4 5 6 7 8

C: 3 3

1 2 3 40 5

2 2 7 7 8 8

3 3

3 3

0 0

0 0

0

1

3 3

3 3

5 5

2 2

Page 36: Counting sort(Non Comparison Sort)

Executing Loop 4

A:

B:

1 2 3 4 5

6 7 8

5 5

1 2 3 4 5 6 7 8

C:

1 2 3 40 5

2 2 7 7 8 8

3 3

3 3

0 0

0 0

0

0 0

3 3

3 3

5 3 3

0 0 2 2

3

Page 37: Counting sort(Non Comparison Sort)

Executing Loop 4

A:

B:

1 2 3 4 5 6 7 8

1 2 3 4 5 6

7 8

C:

1 2 3 40 5

2 2 7 7 8

3 3

3 3

0 0

0 0

0 3 3

3 3

3 3

0 0 2 2

3 3 5

4 4

3 3

0 0

Page 38: Counting sort(Non Comparison Sort)

Executing Loop 4

A:

B:

1 2 3 4 5 6 7 8

1 2 3 4 5 6

7 8

C:

1 2 3 40 5

2 2 7 7 7 7

3 3

3 3

0 0

0 0

0 3 3

3 3

3

0 0 2 2

3 3 2

3 3

0 0

5 5

4 4

5 5

Page 39: Counting sort(Non Comparison Sort)

End of Loop 4

A:

B:

1 2 3 4 5 6 7 8

1 2 3 4 5 6 7 8

C:

1 2 3 40 5

2 2 7 7

3 3

3 3

0 0

0 0

0 3 3

3 3 0 0 2 2

3 3

3 3

0 0

5 5

4 4

5 5

2 2 7 7

2 2

Sorted data in Array B

Page 40: Counting sort(Non Comparison Sort)

Time Complexity Analysis1. Counting-Sort(A, B, k)

2. Let C[0…..k] be a new array

3. for i=0 to k [Loop 1]4. C[i]= 0;

5. for j=1 to A.length or n [Loop 2]6. C[ A[j] ] = C[ A[j] ] + 1;

7. for i=1 to k [Loop 3]8. C[i] = C[i] + C[i-1];

9. for j=n or A.length down to 1 [Loop 4]10. B[ C[ A[j] ] ] = A[j];

11. C[ A[j] ] = C[ A[j] ] - 1;

Loop 2 and 4 takes O(n) time

Loop 1 and 3 takes O(k) time

Page 41: Counting sort(Non Comparison Sort)

Time Complexity Analysis

• So the counting sort takes a total time of: O(n + k)• Counting sort is called stable sort.– A sorting algorithm is stable when numbers with

the same values appear in the output array in the same order as they do in the input array.

Page 42: Counting sort(Non Comparison Sort)

Counting Sort Review

• Why don’t we always use counting sort?– Depends on range k of elements.

• Could we use counting sort to sort 32 bit integers? Why or why not?

Page 43: Counting sort(Non Comparison Sort)

Counting Sort Review• Assumption: input taken from small set of numbers of size k• Basic idea: – Count number of elements less than you for each element.– This gives the position of that number – similar to selection

sort.• Pro’s:– Fast– Asymptotically fast - O(n+k)– Simple to code

• Con’s:– Doesn’t sort in place.– Requires O(n+k) extra storage.

Page 44: Counting sort(Non Comparison Sort)

MD. Shakhawat Hossain

Student of Computer Science & Engineering Dept.

University of Rajshahi