Sorting

CS-212

Dick Steflik

Exchange Sorting

• Method : make n-1 passes across the data, on each pass compare adjacent items, swapping as necessary (n-1 compares)

• O(n2)

Exchange Sorting

int ExchangeSort ( int & num[ ] , const int & numel )

{

int i,j,k, moves = 0;

for ( i = 0 ; i < (numel - 1) ; i++ )

for ( j = 0 ; j < (numel - 1) ; j++ )

if ( num [ j ] < num [ j-1 ])

{

temp = num [ j ];

num [ j ] = num [ j - 1 ];

num [ j - 1] = temp;

moves++;

}

return moves;

}

Basic Algorithm ( no improvements)

Exchange Sort

int ExchangeSort ( int & num[ ] , const int & numel )

{

int i,j,k, moves = 0;

for ( i = 0 ; i < (numel - 1) ; i++ )

for ( j = i ; j < numel ; j++ )

if ( num [ j ] < num [ j-1 ])

{

temp = num [ j ];

num [ j ] = num [ j - 1 ];

num [ j - 1] = temp;

moves++;

}

return moves;

}

Improvement #1 : don’t revist places that are already in the correct place

Exchange Sortint ExchangeSort ( int & num[ ] , const int & numel )

{ int i,j,k, swap,moves = 0;

swap = 1;

for ( i = 0 ; i < (numel - 1) ; i++ )

if ( swap != 0 )

{ swap = 0;

for ( j = 0 ; j < numel ; j++ )

if ( num [ j ] < num [ j-1 ])

{

temp = num [ j ];

num [ j ] = num [ j - 1 ];

num [ j - 1] = temp;

moves++;

swap = 1;

}

}

return moves;

}

// Improvement # 2:

// if a pass through the inner loop occurs

// that doesn’t product a swap, you’re done.

// both of the improvements improve the

// overall performance but to not improve

// O(n2)

4

Exchange Sort

3 7 5 2 4 compare 3 and 7 ; 7 is > 3 so advance

3 5 7 2 4 compare 7 and 5, 7 > 5 so swap them

3 5 2 7 compare 7 and 2, 7 >4 so swap them

3 5 2 4 7 compare 7 and 4, 7 >4 so swap them

End of pass 1; notice that 7 is in the right place

Exchange Sort - pass 2

3 5 2 4 7 compare 3 and 5 ; 3 is >5 so advance

3 2 5 4 7 compare 5 and 2, 5 < 2 so swap them

3 2 4 5 7 compare 5 and 4, 5 > 4 so swap them

3 2 4 5 7 compare 5 and 7, 5 < 7 so pass 2 done

End of pass 2; notice that 5 and 7 are in the right places

3 5 2 4 7

Exchange Sort - pass 3

2 3 4 5 7 compare 3 and 2 ; 3 is >2 so swap them

2 3 4 5 7 compare 3 and 4, 3 <42 so advance

2 3 4 5 7 compare 4 and 5, 4 < 5 so advance

2 3 4 5 7 compare 5 and 7, 5 < 7 so pass 3 done

End of pass 3; notice that 4, 5 and 7 are in the right places

3 2 4 5 7

Exchange Sort - pass n-1 (4)

2 3 4 5 7 compare 2 and 3 ; 2 is < 3 so advance

2 3 4 5 7 compare 3 and 4, 3 <4 so advance

2 3 4 5 7 compare 4 and 5, 4 < 5 so advance

2 3 4 5 7 compare 5 and 7, 5 < 7 so pass n-1 done

End of pass 4; notice that everything is where it should be.

2 3 4 5 7

Exchange Sort Improvements

• on each successive pass, do one less compare, because the last item from that pass is in place

• if you ever make a pass in which no swap occurs, the sort is complete

• These will both improve performance but Big O will remain O(n2)

Insertion Sort

• Strategy: divide the collection into two lists, one listed with one element (sorted) and the other with the remaining elements.

• On successive passes take an item from the unsorted list and insert it into the sorted list so the the sorted list is always sorted

• Do this until the unsorted list is empty

Insertion Sort

3 7 5 2 4

sorted unsorted

take an item from the unsorted list (7) and insert into the sorted list

3 7 5 2 4

sorted unsorted

take next item from the unsorted list (5) and insert into the sorted list

3 5 7 2 4

sorted unsorted

2 3 5 7 4

sorted unsorted

2 3 4 5 7

sorted unsorted

take next item from the unsorted list (2) and insert into the sorted list

take next item from the unsorted list (4) and insert into the sorted list

Insertion Sort

Void InsertionSort ( int A[ ] , int n )

{

int i , j;

int temp;

for ( i = i < n , i++ )

{ // scan down list looking for correct place to put new element

j = i ;

temp = A[ i ];

while ( j < 0 && temp < A[ j-1 ])

{ // shift the list 1 element to the right to make room for new element

A[ j ] = A[ j+1 ];

j--;

}

A [ j ] = temp;

}

}

Insertion Sort

• Note that each insertion could be O(n-1) and there are n-1 insertions being done therefore Big O is O(n2)

• This is very much like building an ordered linked list except there is more data movement

Selection Sort

• Strategy: make a pass across the data looking for the largest item, swap the largest with the last item in the array.

• On successive passes (n-1) assume the array is one smaller (the last item is in the correct place) and repeat previous step

Selection Sortint SelectionSort ( int num [ ] , int numelem )

{ int i , j , min minidx , temp , moves = 0;

for ( i = 0 ; i < numelem ; i++)

{ min = num [ i ] ;

minidx = i ;

for ( j = i + 1 ; j < numelem ; j++ )

{ min = num[ j ] ; minidx = j ; }

if ( min < num[ i ] )

{ temp = num[ i ] ;

num[ i ] = min ;

num[ minidx ];

moves++;

}

}

Selection Sort3 7 5 2 4

biggest last

3 4 5 2 7

3 4 5 2

biggest last

3 4 2 5

7

3 4 2 5

7

7

biggest last

3 2 4

3 2

2 3

5 7

5 74

5 74

Selection Sort

• Notice that in selection sort, there is the least possible data movement

• There are still n-1 compares on sublists that become one item smaller on each pass so, Big O is still O(n2)

• This method has the best overall performance of the O(n2) algorithms because of the limited amount of data movement

Heap Sort

• A heap is a tree structure in which the key at the root is max(keys) and the key of every parent is greater than the key of either of its children

• Visualize your array as a complete binary tree

• Arrange the tree into a heap by recursively reapplying the heap definition

Heap Sort

• repeat the following n-1 times– swap the root with the last element in the tree– starting at the root reinforce the heap

definition– decrement the index of the last element

A Sample Heap

8

5 7

3 2

Notice: largest key is at root

Notice: Keys of children are < key of parent

Heap Sort

void HeapSort( int A[ ] , int n )

{ // this constructor turns the array into a max heap

Heap H(A,n);

int elt;

for (int i = n-1 ; i >= 1 ; i--)

{

// delete smallest element from heap and place it in A[ i ]

elt = H.Hdelete ( ); // Hdelete deletes the largest element and reenforces the heap property

A[ I ] = elt ;

}

}

Visualize array to sort as tree

3 7 5 2 4 3

7 5

2 4

0 1 2 3 40

1 2

3 4

Make into a heap

3

7 5

2 4

0

1

3 4

7

3 5

2 4

0

1

3 4

7

4 5

2 3

0

1

3 4

STEP 1 STEP 2 STEP 3

Start at root, compare root to the children, swap root with largest child

Original Array Repeat on preorder traversal path. At this point we have a heap

Note: Larger arrays will take more steps

Now the sort starts

7

4 5

2 3

0

1

3 4

3

4 5

2 7

0

1

3 4

Swap the root and the last node

Notice what is left is no longer a heap

5

4 3

2 7

0

1

3 4

Re heapize it by swaping the root with its largets child

2

32

2

Sorting...

2

4 3

5 7

0

1

3 4

2

Swap the root and the last node

Notice what is left is no longer a heap

4

2 3

5 7

1

3 4

2

Reenforce the heap property on the remaining tree

0

Still Sorting...

3

2 4

5 7

1

3 4

2

Swap the root with the last element

0

3

2 4

5 7

1

3 4

2

Reenforce the heap property if necessary

02

3 4

5 7

1

3 4

2

Swap the root with the last element

0

And what we have left is...

2 3 4 5 7

0 1 2 3 4

2

3 4

5 7

1

34

2

0

The Merge Principle• Assume there exist two sorted lists (with queue behavior)

7 5 3 1

8 6 4 2

7 5 3

1

8 6 4 2

Compare the items at the front of the list and move the smaller to the back of a third list

Do it again

7 5 3

1

8 6 4

2

The Merge Principle

7 5

3 18 6 4

2

And again, and again, and again…until

And again

7 5 3 18 6 4 2

We have a list whose length is the sum of the lengths and contains all of the elements ofboth lists, and this list is also ordered

Merge Sort

void MergeSort ( int data[ ] , int n )

{

int n1 , n2 ; // size of first subarray and second subarrays respectively

if (n > 1 )

{

n1 = n / 2 ;

n2 = n - n1;

MergeSort ( data , n1); // sort from data[0] to data[n1-1]

MergeSort ((data + n1) , n2); // sort from data[n1] to end

// merge the two sorted halves

Merge (data , n1 , n2 );

}

}

Merge Sort

75 17 5 12 19 24 4 2782314333113443

Picture the given list as a collection of n, 1 element sorted lists (i.e. 75 is a sorted 1 element list, as is 17. Now merge the adjacent lists of length 1 into lists of length 2….

17 75 5 12 19 24 4 2782314333341143

...now merge the lists of length 2 into lists of length 4

5 17 12 75 4 19 24 2723148343311343

...now merge the lists of length 4 into lists of length 8

4 5 12 17 19 24 43 3433272314118375

...now merge the lists of length 8 into lists of length 16

3 4 5 8 11 12 14 754334332723231917

…and there you have it merge sort

Quick Sort

• This sorting method by far outshines all of the others for flat out speed

• Average run time is O(nlog2n)

• there are problems, worst case performance is O(n2) when data is already in sorted order or is almost in sorted order (we’ll analyze this separately)

• and there are solutions to the problems• and there is an improvement to make it faster still

Quick Sort

23 17 5 12 19 24 4 2782614333113443

23 17 5 12 19 244 278 2614333113443

23 17 5 12 19 244 278 2614 3331134 43

23 17 5 12 19 244 278 2614 333 11 34 43

2317 5 12 19 244 278 2614 33311 34 43

Note : 23 is now in the right place and everything to its left is < 23 and everything to its left is > 23

Pick the leftmost element as the pivot (23). Now , start two cursors (one at either end) going towards the middle and swap values that are > pivot (found with left cursor) with values < pivot (found with right cursor)

swap

swap

swap

swap

Finally, swap the pivot and the value where the cursors passed each other

Quick Sort

2317 5 12 19 244 278 2614 33311 34 43

Now, repeat the process for the right partition

17 5 12 19 48 14 311

swap

175 12194 8 14311

swap

175 12194 8 14311

swap

175 121948 143 11

swap

Note: the 11 is now in the right place, and the left partition is all < pivot and the right partition is all > pivot

Quick Sort

175 121948 143 11

Now, repeat the process with the right partition

54 83

Notice that there is nothing to swap , so swap the pivot and the 4, now the 8 is on the right place

Repeat the process on the leftmost partition again

54 3

5 48 3

543The 4 is now in the right place and the left and right partitions are both of size one so they must also be in the right place

Quick Sort

175 12194 8 143 11 23

Now that we’ve exhausted the left partitions, back up and do the right partition

17 12 1914

swap

171214

swap

Since the left partition is just two items, just compare and swap if necessary

12 14

175 12 194 8 143 11 23

Now back up and do the remaining right partition

Quick Sort

175 12 194 8 143 11 23 24 27263334 43

2427 263334 43

swap

swap

24 27 2633 34 43

24 2726 33

swap

2726 33

24 2726 33

175 12 194 8 143 11 23 24 2726 33 34 43

All done

Quick Sort (worst case)

• If the data is already sorted watch what happens to the partitions

175 12 194 8 143 11 23 24 2726 33 34 43

There is nothing to swap

175 12 194 8 1411 23 24 2726 33 34 43

Again, nothing to swap…..

The partitions are always the maximum size and the performance degrades to O(n2)

Quick Sort (worst case solution)

• The problem comes from the way we picked the pivot• Pick the pivot a different way

– median-of-three• instead of always picking the leftmost value of the partition use the median

of the first, the middle and the last value in the partition.

Quick Sort Improvement

• Since Quick sort is a recursive algorithm, a lot of time gets eaten up in the recursion involved with sorting the small partitions

• Solution - use a different algorithm for the small partitions – remember for small collections of data the n2 algorithms perform better

than the log2n algorithms

– this is one place where exchange sort excels• if the partition size is 15 or less use exchange (or selection sort)

Radix Sort