Chapter 7 (Part 2) Chapter 7 (Part 2) SorSorting Ating Algorithmslgorithms

Merge Sort

Merge Sort

Basic IdeaExampleAnalysis

http://www.geocities.com/SiliconValley/Program/2864/File/Merge1/mergesort.html

2

Idea

3

Two sorted arrays can be merged in linear time with N comparisons only.

Given an array to be sorted,

consider separately its left half and its right half,

sort them and then merge them.

Characteristics

Recursive algorithm.Runs in O(NlogN) worst-case running time.

Where is the recursion? Each half is an array that can be sorted

using the same algorithm - divide into two, sort separately the left and the right halves, and then merge them.

4

5

Merge Sort Code

6

void merge_sort ( int [ ] a, int left, int right){ if(left < right) {

int center = (left + right) / 2;

merge_sort (a,left, center);

merge_sort(a,center + 1, right);

merge(a, left, center + 1, right); }}

Analysis of Merge Sort

7

Assumption: N is a power of two.

For N = 1 time is constant (denoted by 1)

Otherwise:

time to mergesort N elements =

time to mergesort N/2 elements +

time to merge two arrays each N/2 el.

Recurrence Relation

8

Time to merge two arrays each N/2 elements is linear, i.e. O(N)

 

Thus we have:

 (a) T(1) = 1

(b) T(N) = 2T(N/2) + N

Solving the Recurrence Relation

9

T(N) = 2T(N/2) + N divide by N:

(1) T(N) / N = T(N/2) / (N/2) + 1

Telescoping: N is a power of two, so we can write

(2) T(N/2) / (N/2) = T(N/4) / (N/4) +1

(3) T(N/4) / (N/4) = T(N/8) / (N/8) +1

…….

T(2) / 2 = T(1) / 1 + 1

Adding the Equations

10

The sum of the left-hand sides will be equal to the sum of the right-hand sides:

T(N) / N + T(N/2) / (N/2) + T(N/4) / (N/4) +

… + T(2)/2 =

T(N/2) / (N/2) + T(N/4) / (N/4) + ….

+ T(2) / 2 + T(1) / 1 + LogN

 (LogN is the sum of 1’s in the right-hand sides)

Crossing Equal Terms, Final Formula

11

After crossing the equal terms, we get

  T(N)/N = T(1)/1 + LogN

 

T(1) is 1, hence we obtain

  T(N) = N + NlogN = (NlogN)

 Hence the complexity of the Merge Sort algorithm is (NlogN).

Quick Sort

Basic IdeaCodeAnalysisAdvantages and DisadvantagesApplicationsComparison with Heap sort and

Merge sort

http://math.hws.edu/TMCM/java/xSortLab/

12

Basic Idea

Pick one element in the array, which will be the pivot.

Make one pass through the array, called a partition step, re-arranging the entries so that: • entries smaller than the pivot are to the left of

the pivot. • entries larger than the pivot are to the right

13

Basic Idea

Recursively apply quicksort to the part of the array that is to the left of the pivot, and to the part on its right.

No merge step, at the end all the elements are in the proper order

14

Choosing the Pivot

15

Some fixed element: e.g. the first, the last, the one in the middle.

Bad choice - may turn to be the smallest or the largest element, then one of the partitions will be empty

Randomly chosen (by random generator) - still a bad choice

Choosing the Pivot

16

The median of the array

(if the array has N numbers, the median is the [N/2] largest number).

This is difficult to compute - increases the complexity.

Choosing the Pivot

17

The median-of-three choice:

take the first, the last and the middle element.

Choose the median of these three elements.

Find the Pivot – Java Code

18

int median3 ( int [ ]a, int left, int right)

{ int center = (left + right)/2;

if (a [left] > a [ center]) swap (a [ left ], a [center]);

if (a [center] > a [right]) swap (a[center], a[ right ]);

if (a [left] > a [ center]) swap (a [ left ], a [center]);

swap(a [ center ], a [ right-1 ]);

return a[ right-1 ];

}

Quick Sort – Java Code

19

If ( left + 10 < = right)

{ // do quick sort }

else insertionSort (a, left, right);

Quick Sort – Java Code

20

{ int i = left, j = right - 1;

for ( ; ; )

{ while (a [++ i ] < pivot) { }

while ( pivot < a [- - j ] ) { }

if (i < j) swap (a[ i ], a [ j ]);

else break; }

swap (a [ i ], a [ right-1 ]);  quickSort ( a, left, i-1);  quickSort (a, i+1, right); }

Implementation Notes

21

Compare the two versions:

 A. while (a[++i] < pivot) { }

while (pivot < a[--j]){ }

if ( i < j ) swap (a[i], a[j]);else break;

 B. while (a[ i ] < pivot) { i++ ; }

while (pivot < a[ j ] ) { j- - ; }

  if ( i < j ) swap (a [ i ], a [ j ]);else break;

Implementation Notes

22

If we have an array of equal elements, the second code will

never increment i or decrement j,

and will do infinite swaps.

i and j will never cross.

Complexity of Quick Sort

23

Average-case O(NlogN)

Worst Case: O(N2)

This happens when the pivot is the smallest (or the largest) element.

Then one of the partitions is empty, and we repeat recursively the procedure for N-1 elements.

Complexity of Quick Sort

24

Best-case O(NlogN)The pivot is the median of the array, the left and the right parts have same size.There are logN partitions, and to obtain each partitions we do N comparisons (and not more than N/2 swaps). Hence the complexity is O(NlogN)

Analysis

25

T(N) = T(i) + T(N - i -1) + cN

The time to sort the file is equal to

the time to sort the left partition with i elements, plus

the time to sort the right partition with N-i-1 elements, plus

the time to build the partitions.

Worst-Case Analysis

26

The pivot is the smallest (or the largest) element

 T(N) = T(N-1) + cN, N > 1

Telescoping:

  T(N-1) = T(N-2) + c(N-1)

T(N-2) = T(N-3) + c(N-2)

T(N-3) = T(N-4) + c(N-3) …………... 

T(2) = T(1) + c.2

Worst-Case Analysis

27

T(N) + T(N-1) + T(N-2) + … + T(2) =

= T(N-1) + T(N-2) + … + T(2) + T(1) +

c(N) + c(N-1) + c(N-2) + … + c.2

 

T(N) = T(1) +

c times (the sum of 2 thru N)

= T(1) + c (N (N+1) / 2 -1) = O(N2)

Best-Case Analysis

28

The pivot is in the middle

  T(N) = 2T(N/2) + cN

 Divide by N:

  T(N) / N = T(N/2) / (N/2) + c

Best-Case Analysis

29

Telescoping:

T(N) / N = T(N/2) / (N/2) + c

T(N/2) / (N/2) = T(N/4) / (N/4) + c

T(N/4) / (N/4) = T(N/8) / (N/8) + c

……

T(2) / 2 = T(1) / (1) + c

Best-Case Analysis

30

Add all equations:

T(N) / N + T(N/2) / (N/2) + T(N/4) / (N/4) + …. + T(2) / 2 =

= (N/2) / (N/2) + T(N/4) / (N/4) + … + T(1) / (1) + c.logN

After crossing the equal terms:

T(N)/N = T(1) + c*LogN

T(N) = N + N*c*LogN = O(NlogN)

Advantages and Disadvantages

Advantages:One of the fastest algorithms on average

Does not need additional memory (the sorting takes place in the array - this is called in-place processing )

Disadvantages:The worst-case complexity is O(N2)

31

Applications

Commercial applications

QuickSort generally runs fast No additional memory The above advantages compensate

for the rare occasions when it runs with O(N2)

32

Applications

Never use in applications which require a guarantee of response time:

Life-critical (medical monitoring, life support in aircraft, space craft)

Mission-critical (industrial monitoring and control in handling dangerous materials, control for aircraft, defense, etc)

unless you assume the worst-case response time

33

Warning:

Comparison with Heap Sort

O(NlogN) complexity quick sort runs faster, (does not

support a heap tree) the speed of quick sort is not

guaranteed

34

Comparison with Merge Sort

Merge sort guarantees O(NlogN) time Merge sort requires additional memory

with size N Usually Merge sort is not used for main

memory sorting, only for external memory sorting

35