Post on 03-Jan-2016
Chapter 7: Chapter 7: Sorting Sorting AAlgorithmslgorithms
Insertion Sort
Sorting Algorithms
Insertion SortShell SortHeap SortMerge SortQuick Sort
2
Assumptions
Elements to sort are placed in arrays of length N
Can be compared
Sorting can be performed in main memory
3
Complexity
Simple sorting algorithms : O(N2)Shellsort: o(N2)Advanced sorting algorithms: O(NlogN)In general: Ω(NlogN)
4
Insertion Sort
5
PRE: array of N elements (from 0 to N-1)
POST: array sorted
1. An array of one element is sorted
2. Assume that the first p elements are sorted.
for j = p to N-1
Take the j-th element and find a place for it among the first j sorted elements
Insertion Sort
6
int j, p; comparable tmp;
for ( p = 1; p < N ; p++) {
tmp = a[p];
for ( j=p; j>0 && tmp < a[ j-1 ] ; j- - )
a[ j ] = a[ j-1 ];
a[ j ] = tmp; }
http://www.cs.auckland.ac.nz/software/AlgAnim/sorting.html#bubble
Analysis of the Insertion Sort
7
Insert the N-th el.: at most N-1 comparisons N-1 movements
Insert the N-1st el. at most N-2 comparisons N-2 movements
Insert the 2nd el.: 1 comparison 1 movement
2*(1 + 2 +… N - 1) = 2*N * (N-1) / 2 =
N(N-1) = Θ (N2) Almost sorted array: O(N) Average complexity: Θ (N2)
A lower bound for simple sorting algorithms
8
• An inversion :
an ordered pair (Ai, Aj) such that
i < j but Ai > Aj
Example: 10, 6, 7, 15, 3,1 Inversions are: (10,6), (10,7), (10,3), (10,1), (6,3), (6,1)(7,3), (7,1) (15,3), (15,1), (3,1)
Swapping
9
Swapping adjacent elements that are out of order removes one inversion. A sorted array has no inversions.
Sorting an array that contains i inversions requires at least i swaps of adjacent elements
How many inversions are there in an average unsorted array?
Theorems
10
Theorem 1: The average number of inversions in an array of N distinct elements is
N (N - 1) / 4
Theorem 2: Any algorithm that sorts by exchanging adjacent elements requires Ω (N2) time on average.
For a sorting algorithm to run in less than quadratic time it must do something other than swap adjacent elements
Shell Sort
Shell Sort
Improves on insertion sort
Compares elements far apart, then less far apart, finally compares adjacent elements (as an insertion sort).
12
Idea
arrange the data sequence in a two-dimensional array
sort the columns of the array
repeat the process each time with smaller number of columns
13
Example
14
3 7 9 0 5 1 6 8 4 2 0 6 1 5 7 3 4 9 8 2
it is arranged in an array with 7 columns (left),
then the columns are sorted (right): 3 7 9 0 5 1 6 3 3 2 0 5 1 5
8 4 2 0 6 1 5 7 4 4 0 6 1 6
7 3 4 9 8 2 8 7 9 9 8 2
Example (cont)
15
3 3 2 0 0 1
0 5 1 1 2 2
5 7 4 3 3 4
4 0 6 4 5 6
1 6 8 5 6 8
7 9 9 7 7 9
8 2 8 9
one column in the last step –
it is only a 6, an 8 and a 9 that have to move a little bit to their correct position
Implementation
16
• one-dimensional array that is
indexed appropriately.
• an increment sequence
to determine how far apart elements to be sorted are
Increment sequence
17
Determines how far apart elements to be sorted are:
h1, h2, ..., ht with h1 = 1
hk-sorted array - all elements spaced a distance hk apart are sorted relative to each other.
Correctness of the algorithm
18
Shellsort only works because an array
that is hk-sorted remains hk-sorted
when hk- 1-sorted.
Subsequent sorts do not undo the work done by previous phases.
The last step (with h = 1) -
Insertion Sort on the whole array
Java code for Shell sort
19
int j, p, gap; comparable tmp;
for (gap = N/2; gap > 0; gap = gap/2)
for ( p = gap; p < N ; p++) { tmp = a[p];
for ( j = p ; j >= gap && tmp < a[ j- gap ];
j = j - gap) a[ j ] = a[ j - gap ];
a[j] = tmp;
}
Increment sequences
20
1. Shell's original sequence:
N/2 , N/4 , ..., 1 (repeatedly divide by 2).
2. Hibbard's increments:
1, 3, 7, ..., 2k - 1 ;
3. Knuth's increments:
1, 4, 13, ..., ( 3k - 1) / 2 ;
4. Sedgewick's increments:
1, 5, 19, 41, 109, ....
9 ·4k - 9 ·2k + 1 or 4k - 3 ·2k + 1.
Analysis
21
Shellsort's worst-case performance using Hibbard's increments is Θ(n3/2).
The average performance is thought to be about O(n 5/4)
The exact complexity of this algorithm is still being debated .
for mid-sized data : nearly as well if not better than the faster (n log n) sorts.
Heap Sort
Heap Sort
Basic IdeaComplexityExample
http://www.cs.auckland.ac.nz/software/AlgAnim/heapsort.html
23
Idea
Store N elements in a binary heap tree.Perform delete_Min operation N times,
storing each element deleted from the heap into another array.
Copy back the array.
• Not very efficient to use two arrays.• Improvement – use one array for the binary heap
and the sorted elements
24
Improvements
Use the same array to store the deleted elements instead of using another array
After each deletion we get a vacant position in the array - the last cell.
There we store the deleted element, which becomes part of the sorted sequence.
25
Improvements
When all the elements are deleted and stored in the same array following the above method, the elements will be there in reversed order.
What is the remedy for this?Store the elements in the binary heap tree in
reverse order of priority - then at the end the elements in the array will be in correct order.
26
Complexity
Sorts in O(NlogN) time by performing N times deleteMax operations.- Each deleteMax operation takes log N running
time.- N times performing deleteMax NlogN
running time
Used for general purpose sorting, guarantees O(N logN)
27
Example
28
15 19 10 7 17 16
1. Consider the values of the elements as
priorities and build the heap tree.
2. Start deleteMax operations, storing
each deleted element at the end of the
heap array.
Example (cont)
29
Note that we use only one array , treating its parts differently:
when sorting, part of the array will be the heap, and the rest part - the sorted array
Build the Heap
30
We start with the element at position SIZE/2comparing the item with the children.The hole is percolated down to position 6 and the item is inserted there.
15 19 7 17 16
10
Result:
15 19 16 7 17 10
hole child
Build the Heap
31
Next we compare position 2 with its children.
15 16 7 17 10
19
hole child1 child2
19 is greater than 7 and 17,
and we continue with position 1
15 19 16 7 17 10
Build the Heap
32
Percolate down the hole at position 1
19 16 7 17 10
15
The hole at position 1 is percolated down to position 2 -the greater child.
19 16 7 17 10
15
Build the Heap
33
Percolate down the hole at position 2
19 16 7 17 10
15
One of the children of the hole at position 2 - item 17, is greater than 15.
So we percolate the hole to position 5.19 17 16 7 15 10
Build the Heap
34
19 17 16 7 15 10
19
17 16
7 15 10
the heap is built
Sorting
35
DeleteMax the top element 19
17 16 7 15 19
17 16
7 15
Store the last heap element (10) in a temporary place.
Move the DeletedMax element (19) to the place where the last heap element was - the current available position in the sorted portion of the array.A hole is created at the top
10
Sorting
36
Percolate down the hole
16 7 15 19
16
7 15
10
17
17
Sorting
37
Percolate down the hole
16 715 19
16
7
15
10
17
17
Sorting
38
Fill the hole
16 715 19
10
16
7
15
1017
17
Sorting
39
DeleteMax the top element 17
16 715 19
16
7
15
17
10
Store the last heap element (10) in a temporary place.
Move the DeletedMax element (17) to the place where the last heap element was - the current available position in the sorted portion of the array.A hole is created at the top
Sorting
40
Percolate down the hole
16 715 19
16
7
15
17
10
Sorting
41
Fill the hole
16 715 19
10
16
7
15
1710
Sorting
42
DeleteMax the top element 16
1615 19
1015
1710
Store the last heap element (7) in a temporary place.
Move the DeletedMax element (16) to the place where the last heap element was - the current available position in the sorted portion of the array.A hole is created at the top
7
Sorting
43
Percolate down the hole
1615 19
10
15
1710
7
Sorting
44
Fill the hole
1615 19
107
15
17107
Sorting
45
DeleteMax the top element 15
1615 19
7
17
10
7
Store the last heap element (10) in a temporary place.
Move the DeletedMax element (15) to the place where the last heap element was - the current available position in the sorted portion of the array.A hole is created at the top
Sorting
46
Percolate down the hole
1615 19
7
17
10
7
Since 10 is greater than the children of the hole, It has to be inserted in the hole
Sorting
47
Fill the hole
1615 19
7
10
1710 7
Sorting
48
DeleteMax the top element 10
1615 1917
7
Store the last heap element (7) in a temporary place.
Move the DeletedMax element (10) to the place where the last heap element was - the current available position in the sorted portion of the array.A hole is created at the top
10
Sorting
49
Fill the hole
1615 19
7
177 10
The hole has no children and so it has to be filled.
Sorted array
50
1615 19177 10
7 is the last element from the heap,
so now the array is sorted