Download - Insertion Sort, Merge Sort and Quicksort

8/10/2019 Insertion Sort, Merge Sort and Quicksort

1/82

1

Sorting

Insertion sort - Mergesort -

Quicksort


2/82

2

Insertion Sort

Algorithm: Move left-to-right through array.

Exchange next element with largerelements to its left, one-by-one.


3/82

3

Example


4/82

4

InsertionSort( ElementType A[ ], int N)

{

int j, P;

Element Type Tmp;

for (P=1; P0 && A[j-1]>Tmp; j--)

A[j] = A[j-1];

A[j] = Tmp;}

}


5/82

5


6/82

6

Contd.....


7/82

7

Analyzing Insertion Sort T(n)=c1n + c2(n-1) + c3(n-1) + c4T + c5(T - (n-1)) + c6(T - (n-1)) + c7(n-1)

= c8T + c9n + c10

What can T be?

Best case -- inner loop body never executed ti= 1T(n) is a linear function

Worst case -- inner loop body executed for all previouselements ti= iT(n) is a quadratic function

Average case ???

Running time for insertion sort is O(n2)


8/82

8

Analysis Simplifications

Ignore actual and abstract statement costs

Order of growthis the interesting measure:

Highest-order term is what counts

Remember, we are doing asymptotic analysis

As the input size grows larger it is the high order term that

dominates


9/82

9

Merge Sort Apply divide-and-conquerto sorting problem

Problem:Given nelements, sort elements intonon-decreasing order

Divide-and-Conquer:

If n=1 terminate (every one-element list is alreadysorted)

If n>1, partition elements into two or more sub-

collections; sort each; combine into a single sorted list How do we partition?


10/82

10

Partitioning - Choice 1 First n-1 elements into set A, last element set B

Sort A using this partitioning scheme recursively B already sorted

Combine A and B using method Insert() (= insertion

into sorted array)

Leads to recursive version of InsertionSort()

Number of comparisons: O(n2)

Best case = n-1

Worst case =2

)1(

2

==

nnic

n

i


11/82

11

Partitioning - Choice 2 Put element with largest key in B, remaining elements

in A Sort A recursively

To combine sorted A and B, append B to sorted A

Use Max() to find largest elementrecursiveSelectionSort()

Use bubbling process to find and move largest element to

right-most positionrecursive BubbleSort() All O(n2)


12/82

12

Partitioning - Choice 3

Lets try to achieve balanced partitioning A gets n/2 elements, B gets rest half

Sort A and B recursively Combine sorted A and B using a process

called merge, which combines two sorted

lists into one How? We will see soon


13/82

13


14/82

14

14 23 45 98 6 33 42 67

MergeSort


15/82

15Merge

23 45 98 33 42 6714 6


16/82

16Merge

23 45 98 6 42 67

6

14 33


17/82

17

Merge

14 45 98 6 42 67

6 14

23 33


18/82

18

Merge

14 23 98 6 42 67

6 14 23

45 33


19/82

19

Merge

14 23 98 6 33 67

6 14 23 33

45 42


20/82

20

Merge

14 23 98 6 33 42

6 14 23 33 42

45 67


21/82

21

Merge

14 23 45 6 33 42

6 14 23 33 42 45

98 67


22/82

22

Merge

14 23 45 98 6 33 42 67

6 14 23 33 42 45 67


23/82

23

Merge

14 23 45 98 6 33 42 67

6 14 23 33 42 45 67 98


24/82

24

void mergeSort(Comparable A[], int left, int right)

{

// sort a[left:right]

if (left < right)

{ // at least two elements

int mid = (left+right)/2; //midpoint

mergeSort(A, left, mid);

mergeSort(A, mid + 1, right);

merge(A, B, left, mid, right); //merge from A to B

copy(B, A, left, right); //copy result back to A

}

}


25/82

25

Evaluation

Recurrence equation: Assume n is a power of 2

c1 if n=1

T(n) =

2T(n/2) + c2n if n>1, n=2k


26/82

26

SolutionBy Substitution:T(n) = 2T(n/2) + c2n

T(n/2) = 2T(n/4) + c2n/2

T(n) = 4T(n/4) + 2 c2n

T(n) = 8T(n/8) + 3 c2n

T(n) = 2iT(n/2i) + ic2

n

Assuming n = 2k, expansion halts when we get T(1) on right side; this happenswhen i=k T(n) = 2kT(1) + kc2n

Since 2k=n, we know k=logn; since T(1) = c1, we get

T(n) = c1n + c2nlogn;

Thus an upper bound for TmergeSort(n) isO(nlogn)


27/82

27

Quicksort Algorithm

Given an array of nelements (e.g., integers): If array only contains one element, return

Else

pick one element to use as pivot.

Partition elements into two sub-arrays:

Elements less than or equal to pivot

Elements greater than pivot Quicksort two sub-arrays

Return results


28/82

28

ExampleWe are given array of n integers to sort:

40 20 10 80 60 50 7 30 100


29/82

29

Pick Pivot ElementThere are a number of ways to pick the pivot element. In this

example, we will use the first element in the array:

40 20 10 80 60 50 7 30 100


30/82

30

Partitioning Array

Given a pivot, partition the elements of the array

such that the resulting array consists of:1. One sub-array that contains elements >= pivot

2. Another sub-array that contains elements < pivot

The sub-arrays are stored in the original data array.

Partitioning loops through, swapping elementsbelow/above pivot.


31/82

31

40 20 10 80 60 50 7 30 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]

too_big_index too_small_index


32/82

32

40 20 10 80 60 50 7 30 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]


1. While data[too_big_index]


33/82

33

40 20 10 80 60 50 7 30 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]




34/82

34

40 20 10 80 60 50 7 30 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]




35/82

35

40 20 10 80 60 50 7 30 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]


1. While data[too_big_index] data[pivot]

--too_small_index


36/82

36

40 20 10 80 60 50 7 30 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



--too_small_index


37/82

37

40 20 10 80 60 50 7 30 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



--too_small_index3. If too_big_index < too_small_index

swap data[too_big_index] and data[too_small_index]


38/82

38

40 20 10 30 60 50 7 80 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



--too_small_index3. If too_big_index < too_small_index



39/82

39

40 20 10 30 60 50 7 80 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



--too_small_index

3. If too_big_index < too_small_index

swap data[too_big_index] anddata[too_small_index]

4. While too_small_index > too_big_index, go to 1.


40/82

40

40 20 10 30 60 50 7 80 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



--too_small_index


swap data[too_big_index] and data[too_small_index]4. While too_small_index > too_big_index, go to 1.


41/82

41

40 20 10 30 60 50 7 80 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



--too_small_index




42/82

42

40 20 10 30 60 50 7 80 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



--too_small_index




43/82

43

40 20 10 30 60 50 7 80 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



--too_small_index




44/82

44

40 20 10 30 60 50 7 80 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



--too_small_index




45/82

45


--too_small_index



40 20 10 30 7 50 60 80 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]


1 Whil d t [t bi i d ] d t [ i t]


46/82

46


--too_small_index



40 20 10 30 7 50 60 80 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



47/82

1 While data[too big index] < data[pivot]


48/82

48


--too_small_index



40 20 10 30 7 50 60 80 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



49/82


50/82

1 Whil d t [t bi i d ] < d t [ i t]


51/82

51


--too_small_index



40 20 10 30 7 50 60 80 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



52/82

52


--too_small_index



40 20 10 30 7 50 60 80 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]


1 Whil d [ bi i d ] d [ i ]


53/82

53


--too_small_index



40 20 10 30 7 50 60 80 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]


1 While data[too big index]


54/82

54


--too_small_index



5. Swap data[too_small_index] and data[pivot_index]

40 20 10 30 7 50 60 80 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]


1 While data[too big index]


55/82

55


--too_small_index




7 20 10 30 40 50 60 80 100pivot_index = 4

[0] [1] [2] [3] [4] [5] [6] [7] [8]



56/82

56

Partition Result

7 20 10 30 40 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

data[pivot]


57/82

57

Recursion: Quicksort Sub-arrays

7 20 10 30 40 50 60 80 100

[0] [1] [2] [3] [4] [5] [6] [7] [8]

data[pivot]


58/82

58


59/82

59

Quicksort Analysis

Assume that keys are random, uniformlydistributed.

What is best case running time?


60/82

60

Quicksort Analysis



Recursion:1. Partition splits array in two sub-arrays of size n/2

2. Quicksort each sub-array


61/82

61

Quicksort Analysis





Depth of recursion tree?


62/82

62

Quicksort Analysis





Depth of recursion tree? O(log2n)


63/82

63

Quicksort Analysis






Number of accesses in partition?


64/82

64

Quicksort Analysis






Number of accesses in partition? O(n)


65/82

65

Quicksort Analysis


Best case running time: O(n log2n)


66/82

66

Quicksort Analysis



Worst case running time?


67/82

67

Quicksort: Worst Case

Assume first element is chosen as pivot.

Assume we get array that is already in

order:

2 4 10 12 13 50 57 63 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



68/82

68


--too_small_index





2 4 10 12 13 50 57 63 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



69/82

69


--too_small_index





2 4 10 12 13 50 57 63 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



70/82

70


--too_small_index





2 4 10 12 13 50 57 63 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



71/82

71


--too_small_index





2 4 10 12 13 50 57 63 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



72/82

72


--too_small_index





2 4 10 12 13 50 57 63 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



73/82

73


--too_small_index





2 4 10 12 13 50 57 63 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]



74/82

74


--too_small_index





2 4 10 12 13 50 57 63 100pivot_index = 0

[0] [1] [2] [3] [4] [5] [6] [7] [8]

> data[pivot]


75/82

75

Quicksort Analysis

Assume that keys are random, uniformly

distributed.



Recursion:1. Partition splits array in two sub-arrays:

one sub-array of size 0

the other sub-array of size n-1


Depth of recursion tree?


76/82

76

Quicksort Analysis


distributed. Best case running time: O(n log2n)






Depth of recursion tree? O(n)


77/82

77

Quicksort Analysis









Number of accesses per partition?


78/82

78

Quicksort Analysis









Number of accesses per partition? O(n)


79/82

79

Quicksort Analysis


distributed.


Worst case running time: O(n2

)!!!


80/82

80

Quicksort Analysis


distributed.


Worst case running time: O(n2

)!!! What can we do to avoid worst case?


81/82

81

Improved Pivot Selection

Pick median value of three elements from data array:

data[0], data[n/2], and data[n-1].

Use this median value as pivot.

Improving Performance of


82/82

82

p g

Quicksort

Improved selection of pivot.

For sub-arrays of size 3 or less, apply brute

force search:

Sub-array of size 1: trivial Sub-array of size 2:

if(data[first] > data[second]) swap them

Sub-array of size 3: left as an exercise.