Data Structures 5 Sorting Prof A Alkhorabi. 5- 2 Overview Why Sorting? Simple Sorting – Selection...

Data Structures

5

Sorting

Prof A AlkhorabiProf A Alkhorabi

5- 2Prof A Alkhorabi

Overview

• Why Sorting? • Simple Sorting – Selection & Insertion Sorts• Sorting Performance• Shell Sort• Quicksort• Merge Sort• External Sorting


Why Sorting?• Sorting is important because sorted data can be

searched and merged efficiently.• Problem: Given an unsorted array of data, rearrange the

data into ascending order.• Sorting is a common function in many computer

applications:– Transactions are often processed or reported in some sequence, eg. date.– Files are listed in some sequence, eg. alphabetic order or based on file

size.– Sorting function may be added as a method in a data structure ADT

• Efficient sorting of large quantities of data requires some clever algorithms.

• Some of the most popular search algorithms will be reviewed.


Sorting Files vs. Memory

• Different techniques are often required to sort very large amounts of data stored in a file.

• The cheapness and large capacities of memories means that most sorting can now be done in the memory.

• All the algorithms considered assume that the data to be sorted is held in memory.


Simple Sorting Algorithms

A number of simple sorting algorithms exist:• Bubble Sort, bubbles the largest item to the top.

After n – 1 passes the list will be sorted.• Selection Sort, which works by selecting the

smallest item and placing it in the first position and then continuing this process on the remainder of the list.

• Insertion sort, which works by inserting each item in turn into a partially sorted list.


Bubble (Exchange) Sort

Algorithm1. Start with first element in the list2. Compare this element with next element in the list

If this element is less than next element leave it as it is If not, exchange the two elements locations.

3. Repeat (2) for all other elements4. Repeat (1) to (3) until all elements are sorted.

Number of repetitions required = no of elements - 1


Sorting using bubble sort

Original [15 2 7 4 20 1 10]

First Rep [2 7 4 15 1 10 20]

Sec. Rep [2 4 7 1 10 15 20]

Third Rep [2 4 1 7 10 15 20]

Fourth Rep [2 1 4 7 10 15 20]

Fifth Rep [1 2 4 7 10 15 20]

Bubble (Exchange) Sort


/* Bubble sort: it is an exchange sort, exchanges (swaps) the two adjacent array elements if the second is smaller than the first */

void bubble_sort(int arr[], int elements){int i, j;void swap(int *num1, int *num2);

for(i = 0; i < elements; i++) for(j = 1; j < elements; j++)

if(arr[j] < arr[j - 1]) swap(&arr[j], &arr[j-1]);

}

void swap(int *a, int *b){int num;

num = *a; *a = *b; *b = num;}


void print_array(int *p, int elements){ int i;

printf("[ "); for(i = 0; i < elements; i++) printf("%d ", p[i]); printf("]\n");}


/* sort_bub.c : Bubble (Exchange) sort Prof Abdullah Alkhorabi */

#include <stdio.h>

void main(){ int array[7] = {15,2,7,4,20,1,10}; void bubble_sort(int arr[], int elements),

print_array(int arr[], int elements);

printf("Bubble Sort\n"); printf("===========\n"); printf("List before sorting : "); print_array(array, 7); bubble_sort(array, 7); printf("List after sorting : "); print_array(array, 7);}


/* sort_bub.c Output */

Bubble Sort===========List before sorting : [15 2 7 4 20 1 10]List after sorting : [1 2 4 7 10 15 20]

Example: Sort_Bub.cpp

Time complexity is O(n2).


Selection sort (1)

• Idea: Find the least value in the array, swap it into the leftmost component (where it belongs), and then forget the leftmost component. Do this repeatedly.


Selection sort (2)

• Selection sort algorithm:

To sort a[left…right] into ascending order:

1. For l = left, …, right–1, repeat:1.1. Set p such that a[p] is the least of a[l…right].1.2. If p l, swap a[p] and a[l].

2. Terminate.

• Loop invariant:

lesser values (sorted) greater values (unsorted)

left+1

al right–1left l–1 right


To sort a[left…right] into ascending order:1. For l = left, …, right–1, repeat:

1.1. Set p such that a[p] is the least of a[l…right].1.2. If p l, swap a[p] and a[l].

2. Terminate.

foxleft = 0 1 2 3 4 5 8 = right

cow pig cat rat lion tigera goat dog6 7

Selection sort (3)

• Animation:



2. Terminate.

foxleft = 0 1 2 3 4 5 8 = right

cow pig cat rat lion tiger

0

a

l

goat dog6 7



2. Terminate.

foxleft = 0 1 2 3 4 5 8 = right


0

a

l

goat dog6 7

3p



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right

cow pig fox rat lion tiger

0

a

l

goat dog6 7

3p



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


1

a

l

goat dog6 7



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


1

a

l

goat dog6 7

1p



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


1

a

l

goat dog6 7

1p



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


2

a

l

goat dog6 7



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


2

a

l

goat dog6 7

8p



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right

cow dog fox rat lion tiger

2

a

l

goat pig6 7

8p



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


3

a

l

goat pig6 7



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


3

a

l

goat pig6 7

3p



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


3

a

l

goat pig6 7

3p



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


4

a

l

goat pig6 7



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


4

a

l

goat pig6 7

7p



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right

cow dog fox goat lion tiger

4

a

l

rat pig6 7

7p



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


5

a

l

rat pig6 7



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


5

a

l

rat pig6 7

5p



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


5

a

l

rat pig6 7

5p



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


6

a

l

rat pig6 7



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


6

a

l

rat pig6 7

8p



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right

cow dog fox goat lion pig

6

a

l

rat tiger6 7

8p



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


7

a

l

rat tiger6 7



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


7

a

l

rat tiger6 7

7p



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


7

a

l

rat tiger6 7

7p



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


8

a

l

rat tiger6 7



2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


8

a

l

rat tiger6 7


Selection sort (4)• Analysis (counting comparisons):

Let n = right – left + 1 be the length of the array.

Step 1.1 performs right–l comparisons.This is repeated with l = left, …, right–2, right–1.

No. of comparisons = (right–left) + … + 2 + 1= (n–1) + … + 2 + 1 *= (n – 1)n/2= (n2 – n)/2


* Note that the sum of the series 1, 2, …, n–1, n (where n 0) equals n(n + 1)/2.


Sorting using selection sortOriginal [15 2 7 4 20 1 10]First Rep [1 2 7 4 20 15 10]Sec. Rep [1 2 7 4 20 15 10]Third Rep [1 2 4 7 20 15 10]Fourth Rep [1 2 4 7 20 15 10]Fifth Rep [1 2 4 7 10 15 20]Sixth Rep [1 2 4 7 10 15 20]

Selection Sort


/* Selection sort: selects the lowest value element in the rest of the array, and exchange it with the present element */

void selection_sort(int arr[], int elements)

{

int i, j, minimum, temp;

void swap(int *num1, int *num2);

for(i = 0; i < elements; i++)

{ temp = i;

minimum = arr[i];

for(j = i+1; j < elements; j++)

if(arr[j] < minimum)

{ minimum = arr[j];

temp = j; }

swap(&arr[i], &arr[temp]);

}}


/* sort_sel.c Output */

Selection Sort

===========

List before sorting : [15 2 7 4 20 1 10]

List after sorting : [1 2 4 7 10 15 20]

Example: SortSel.cpp


Insertion sort (1)

• Idea: We can sort a file of values by successively reading each value and inserting it into its correct position in an array. Use the same idea to sort an array of values in place.


Insertion sort (2)

• Insertion sort algorithm:


1. For r = left+1, …, right, repeat:1.1. Let val = a[r].1.2. Insert val into its correct sorted position in a[left…r].

2. Terminate.

• Loop invariant:

inserted (sorted) still to be inserted

left+1

ar right–1left r–1 right


Insertion sort (3)

• Animation:

To sort a[left…right] into ascending order:1. For r = left+1, …, right, repeat:

1.1. Let val = a[r].1.2. Insert val into its correct sorted position in a[left…

r].2. Terminate.

foxleft = 0 1 2 3 4 5 8 = right

cow pig cat rat lion tigera goat dog6 7



r].2. Terminate.

foxleft = 0 1 2 3 4 5 8 = right


1

a

r

goat dog6 7



r].2. Terminate.

cowleft = 0 1 2 3 4 5 8 = right

fox pig cat rat lion tiger

1

a

r

goat dog6 7



r].2. Terminate.

cowleft = 0 1 2 3 4 5 8 = right


2

a

r

goat dog6 7



r].2. Terminate.

cowleft = 0 1 2 3 4 5 8 = right


2

a

r

goat dog6 7



r].2. Terminate.

cowleft = 0 1 2 3 4 5 8 = right


3

a

r

goat dog6 7



r].2. Terminate.

catleft = 0 1 2 3 4 5 8 = right

cow fox pig rat lion tiger

3

a

r

goat dog6 7



r].2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


4

a

r

goat dog6 7



r].2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


4

a

r

goat dog6 7



r].2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


5

a

r

goat dog6 7



r].2. Terminate.

catleft = 0 1 2 3 4 5 8 = right

cow fox lion pig rat tiger

5

a

r

goat dog6 7



r].2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


6

a

r

goat dog6 7



r].2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


6

a

r

goat dog6 7



r].2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


7

a

r

goat dog6 7



r].2. Terminate.

7r



r].2. Terminate.

7r

catleft = 0 1 2 3 4 5 8 = right

cow fox goat lion pig rata tiger dog6 7

catleft = 0 1 2 3 4 5 8 = right

cow fox goat lion pig rata tiger dog6 7



r].2. Terminate.

catleft = 0 1 2 3 4 5 8 = right

cow fox goat lion pig rat

8

a

r

tiger dog6 7



r].2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


8

a

r

rat tiger6 7



r].2. Terminate.

catleft = 0 1 2 3 4 5 8 = right


9

a

r

rat tiger6 7



r].2. Terminate.

catleft = 0 1 2 3 4 5 8 = right

cow dog fox goat lion piga rat tiger6 7


Insertion sort (4)

• Analysis (counting comparisons):

Let n = right – left + 1 be the length of the array.

Step 1.2 performs between 1 and r – left comparisons,say (r – left + 1)/2 comparisons on average.This is repeated with r = left+1, left+2, …, right.

Average no. of comparisons = 2/2 + 3/2 + … + n/2= (n – 1)(n + 2)/4= (n2 + n – 2)/4



Sorting using insersion sortOriginal [15 2 7 4 20 1 10]First Rep [2 15 7 4 20 1 10]Sec. Rep [2 7 15 4 20 1 10]Third Rep [2 4 7 15 20 1 10]Fourth Rep [2 4 7 15 20 1 10]Fifth Rep [1 2 4 7 15 20 10]Sixth Rep [1 2 4 7 10 15 20]


/* Insertion sort : First swaps the first two array elements if the second is smaller, then each element in the rest of the array is inserted within the sorted elements */

void insertion_sort(int arr[], int elements){ int i, j, temp;void shift_array(int array[], int a, int b);

for(i = 1; i < elements; i++) { for(j = 0; j < i; j++)

if(arr[j] > arr[i]) { temp = arr[i]; shift_array(arr, j, i); arr[j] = temp; break;} }

}


/* Shifts the subarray part lies between 'a' and 'b' one position */void shift_array(int *array, int a, int b){ int i;

for(i = b; i >= a; i--) array[i] = array[i-1];}


/* sort_ins.c Output */

Insertion Sort

===========

List before sorting : [15 2 7 4 20 1 10]

List after sorting : [1 2 4 7 10 15 20]

Example: Sort_Ins.cpp


Sorting Performance

• We measure the performance of a sorting algorithm in terms of the number of comparisons that must be done.

• For a Bubble Sort, every element is compared with every other element, so this is O(n2).

• For Selection Sort, the number of comparisons is slightly less, but for a large data set it is still quite close to n2 and we say it is still O(n2).

• The same applies with Insertion Sort except that the inner loop will terminate early if the data is already in order. So for a pre-sorted input, the performance is O(n), but the average performance is still O(n2).


Improved Sorting Algorithms• In order to achieve better than O(n2) performance

it is necessary to swap elements that are not adjacent.

• We following two algorithms do this:– Shellsort (named after Donald Shell) is essentially

an Insertion Sort that looks at items some distance apart. Each pass reduces this distance and the last pass looks at adjacent items (and is therefore an Insertion Sort).

– Quicksort is the fastest known algorithm in practice and works by partitioning the data and recursively sorting smaller partitions.


Quick Sort

• Basic strategy is divide & conquer (recursive):– Pick a random element – the pivot.– Partition the elements into those greater than and

those less than the pivot. This is the difficult part!– Do this recursively on each of the two partitions

while the number of elements in a partition is more than 1.


Quick-sort (1)

• Idea: Choose any value from the array (called the pivot). Then partition the array into three subarrays such that:

– the left subarray contains only values less than (or equal to) the pivot;

– the middle subarray contains only the pivot;

– the right subarray contains only values greater than (or equal to) the pivot.

Finally sort the left subarray and the right subarray separately.

• This is another application of the divide-and-conquer strategy.


Quick-sort (2)

• Quick-sort algorithm:


1. Select a[p]:1.1. Partition a[left…right] such that

a[left…p–1] are all less than or equal to a[p], and a[p+1…right] are all greater than or equal to a[p].

1.2. Sort a[left…p–1] into ascending order.1.3. Sort a[p+1…right] into ascending order.

2. Terminate.


Quick-sort (3)

• Invariants:After step 1.1:

p+1

apleft p–1 right

After step 1.3:

p+1

apleft p–1 right

less than or equal to pivot (unsorted)

pivot greater than or equal to pivot (unsorted)

less than or equal to pivot (sorted)

pivot greater than or equal to pivot (sorted)


Quick-sort (4)

• Animation:To sort a[left…right] into ascending order:1. If left < right:

1.1. Partition a[left…right] such that a[left…p–1] are all less than or equal to a[p], and a[p+1…right] are all greater than or equal to a[p].


2. Terminate.

fox

left = 0 1 2 3 4 5 8 = right

cow pig cat rat lion tigera goat dog

6 7

To sort a[left…right] into ascending order:1. If left < right:



2. Terminate.

fox

left = 0 1 2 3 4 5 8 = right

cow pig cat rat lion tigera goat dog

6 7




2. Terminate.

cow

left = 0 1 2 3 4 5 8 = right

cat dog fox pig rat liona tiger goat

6 7

3p




2. Terminate.

cat

left = 0 1 2 3 4 5 8 = right

cow dog fox pig rat liona tiger goat

6 7

3p




2. Terminate.

cat

left = 0 1 2 3 4 5 8 = right

cow dog fox goat lion piga rat tiger

6 7

3p

To sort a[left…right] into ascending order:1. Select pivot a[p]:



2. Terminate.

cat

left = 0 1 2 3 4 5 8 = right

cow dog fox goat lion piga rat tiger

6 7


Sorting using quick sortOriginal [15 2 7 4 20 1

10]

First Rep [1 2 4 7 20 15 10]

Sec Rep [1 2 4 7 10 15 20]

Quick-sort (4)


Quick-sort (5)

• Analysis (counting comparisons):

Let n be the no. of values to be sorted.Let the total no. of comparisons required to sort n values be comps(n).

Step 1.1 takes about n–1 comparisons to partition the array. (NB: partition, not sort!)


Quick-sort (6)

• In the best case, the pivot turns out to be the median value in the array. So the left and right subarrays both have length about n/2. Then steps 1.2 and 1.3 take about comps(n/2) comparisons each.

Therefore:comps(n) 2 comps(n/2) + n – 1 if n > 1comps(n) = 0 if n 1

Solution:comps(n) n log2n

Best-case time complexity is O(n log n).


Quick-sort (7)

• In the worst case, the pivot turns out to be the smallest value. So the right subarray has length n–1 whilst the left subarray has length 0. Then step 1.3 performs comps(n–1) comparisons, but step 1.2 does nothing at all.

Therefore:comps(n) comps(n–1) + n – 1 if n > 1comps(n) = 0 if n 1

Solution:comps(n) (n2 – n)/2

Worst-case time complexity is O(n2).

• The worst case arises if the array is already sorted!


Quick-sort (8)/* Quick sort : First select a pivot (mean of the first three elements or simply the

first element). Start from the second array element, put all elements of the array less than pivot left of the pivot and the rest to the right of it. Do the same (recursively) to the right and to the left lists. */

void quick_sort(int arr[], int first, int last){ int i, j, pivot, previous, middle, temp;

i = first; j = last;middle = arr[(first+last)/2];do { while(arr[i] < middle && i < last) i++; while(middle < arr[j] && j > first) j--; if(i <= j) { swap(&arr[j], &arr[i]); i++; j--; } }while (i <= j);

if ( first < j ) quick_sort(arr, first, j);if ( i < last ) quick_sort(arr, i, last);}


/* Output of sort_qck.c

[ 15 2 7 4 20 1 10 ]

[ 1 2 4 7 10 15 20 ]

Example: Sort_Qck.cpp


Quick-sort Performance• The performance is O(n log n) if the algorithm is coded

carefully. A bad choice of pivot can result in O(n2) performance.

• Suppose that sorting 1,000 items takes 0.001 seconds. What is the big deal about performance? Does it really matter much?

• However, if we were to now sort 1,000,000 items with an O(n2) algorithm like Insertion sort, it would take about 1000 seconds (nearly 17 minutes) .

• If we instead used an O(n log n) algorithm like Quicksort sorting 1,000,000 items would take about 2 seconds – a significant improvement!


Comparison of sorting algorithms

Algorithm No. of comparisons

No. of copies

Time complexity

Space complexity

Selection sort ~ n2/2 ~ 2n O(n2) O(1)

Insertion sort ~ n2/4 ~ n2/4 O(n2) O(1)

Quick-sort ~ n log2n~ n2/2

~ 2n/3 log2n0

O(n log n)O(n2)

O(log n)O(n)

bestworst

Example: AllSortingAlgorithms.cpp


Exercise:

• Write an algorithm and a C++ program that sorts the array of the following array of strings using Quick Sort for strings:["fox","cow","pig","cat","rat","lion","tiger","goat","dog"]

HINT: First select a pivot (mean of the first three

elements or simply the first element). Start from the second

array element, put all elements of the array less than pivot

left of the pivot and the rest to the right of it. Do the

same (recursively) to the right and the left lists.

Data Structures 5 Sorting Prof A Alkhorabi. 5- 2 Overview Why Sorting? Simple Sorting – Selection...

Documents

Transcript of Data Structures 5 Sorting Prof A Alkhorabi. 5- 2 Overview Why Sorting? Simple Sorting – Selection...