Sorting Algorithms Merge Sort Quick Sort Hairong Zhao New Jersey Institute of Technology.

82
Sorting Algorithms Merge Sort Quick Sort Hairong Zhao http://web.njit.edu/~hz2 New Jersey Institute of Technology

description

3 Divide and Conquer 1.Base Case, solve the problem directly if it is small enough 2.Divide the problem into two or more similar and smaller subproblems 3.Recursively solve the subproblems 4.Combine solutions to the subproblems

Transcript of Sorting Algorithms Merge Sort Quick Sort Hairong Zhao New Jersey Institute of Technology.

Page 1: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

Sorting Algorithms

Merge Sort Quick Sort

Hairong Zhaohttp://web.njit.edu/~hz2

New Jersey Institute of Technology

Page 2: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

2

Overview Divide and Conquer

Merge Sort

Quick Sort

Page 3: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

3

Divide and Conquer1. Base Case, solve the problem directly

if it is small enough

2. Divide the problem into two or more similar and smaller subproblems

3. Recursively solve the subproblems

4. Combine solutions to the subproblems

Page 4: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

4

Divide and Conquer - SortProblem: Input: A[left..right] – unsorted array of integers Output: A[left..right] – sorted in non-decreasing

order

Page 5: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

5

Divide and Conquer - Sort1. Base case

at most one element (left ≥ right), return

2. Divide A into two subarrays: FirstPart, SecondPart Two Subproblems:

sort the FirstPart sort the SecondPart

3. Recursively sort FirstPart sort SecondPart

4. Combine sorted FirstPart and sorted SecondPart

Page 6: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

6

Overview Divide and Conquer

Merge Sort

Quick Sort

Page 7: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

7

Merge Sort: Idea

Merge

Recursively sort

Divide intotwo halves FirstPart SecondPart

FirstPart SecondPart

A

A is sorted!

Page 8: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

8

Merge Sort: AlgorithmMerge-Sort (A, left, right)

if left ≥ right return

else

middle ← b(left+right)/2

Merge-Sort(A, left, middle)

Merge-Sort(A, middle+1, right)

Merge(A, left, middle, right)

Recursive Call

Page 9: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

9A[middle]A[middle]A[left]A[left]

SortedSorted FirstPartFirstPart

Sorted Sorted SecondPartSecondPart

Merge-Sort: Merge

A[right]A[right]

mergemerge

A:A:

A:A:

SortedSorted

Page 10: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

10

6 10 14 223 5 15 28L:L: R:R:

Temporary ArraysTemporary Arrays

5 15 28 30 6 10 145

Merge-Sort: Merge Example

2 3 7 8 1 4 5 6A:A:

Page 11: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

11

Merge-Sort: Merge Example

3 5 15 28 30 6 10 14

L:L:

A:A:

3 15 28 30 6 10 14 22R:R:

i=0 j=0

k=0

2 3 7 8 1 4 5 6

1

Page 12: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

12

Merge-Sort: Merge Example

1 5 15 28 30 6 10 14

L:L:

A:A:

3 5 15 28 6 10 14 22R:R:

k=1

2 3 7 8 1 4 5 6

2

i=0 j=1

Page 13: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

13

Merge-Sort: Merge Example

1 2 15 28 30 6 10 14

L:L:

A:A:

6 10 14 22R:R:

i=1

k=2

2 3 7 8 1 4 5 6

3

j=1

Page 14: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

14

Merge-Sort: Merge Example

1 2 3 6 10 14

L:L:

A:A:

6 10 14 22R:R:

i=2 j=1

k=3

2 3 7 8 1 4 5 6

4

Page 15: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

15

Merge-Sort: Merge Example

1 2 3 4 6 10 14

L:L:

A:A:

6 10 14 22R:R:

j=2

k=4

2 3 7 8 1 4 5 6

i=2

5

Page 16: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

16

Merge-Sort: Merge Example

1 2 3 4 5 6 10 14

L:L:

A:A:

6 10 14 22R:R:

i=2 j=3

k=5

2 3 7 8 1 4 5 6

6

Page 17: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

17

Merge-Sort: Merge Example

1 2 3 4 5 6 14

L:L:

A:A:

6 10 14 22R:R:

k=6

2 3 7 8 1 4 5 6

7

i=2 j=4

Page 18: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

18

Merge-Sort: Merge Example

1 2 3 4 5 6 7 14

L:L:

A:A:

3 5 15 28 6 10 14 22R:R:

2 3 7 8 1 4 5 6

8

i=3 j=4

k=7

Page 19: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

19

Merge-Sort: Merge Example

1 2 3 4 5 6 7 8

L:L:

A:A:

3 5 15 28 6 10 14 22R:R:

2 3 7 8 1 4 5 6

i=4 j=4

k=8

Page 20: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

20

Merge(A, left, middle, right)1. n1 ← middle – left + 12. n2 ← right – middle3. create array L[n1], R[n2]4. for i ← 0 to n1-1 do L[i] ← A[left +i]5. for j ← 0 to n2-1 do R[j] ← A[middle+j]6. k ← i ← j ← 07. while i < n1 & j < n2 8. if L[i] < R[j] 9. A[k++] ← L[i++]10. else11. A[k++] ← R[j++]12. while i < n1

13. A[k++] ← L[i++]14. while j < n2

15. A[k++] ← R[j++] n = n1+n2

Space: n

Time : cn for some constant c

Page 21: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

21

6 2 8 4 3 7 5 16 2 8 4 3 7 5 1

Merge-Sort(A, 0, 7)Divide

A:

Page 22: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

22

6 2 8 4

3 7 5 1

6 2 8 4

Merge-Sort(A, 0, 3) , divideA:

Merge-Sort(A, 0, 7)

Page 23: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

23

3 7 5 1

8 4

6 26 2

Merge-Sort(A, 0, 1), divideA:

Merge-Sort(A, 0, 7)

Page 24: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

24

3 7 5 1

8 4

6

2

Merge-Sort(A, 0, 0) , base caseA:

Merge-Sort(A, 0, 7)

Page 25: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

25

3 7 5 1

8 4

6 2

Merge-Sort(A, 0, 0), returnA:

Merge-Sort(A, 0, 7)

Page 26: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

26

3 7 5 1

8 4

6

2

Merge-Sort(A, 1, 1), base caseA:

Merge-Sort(A, 0, 7)

Page 27: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

27

3 7 5 1

8 4

6 2

Merge-Sort(A, 1, 1), returnA:

Merge-Sort(A, 0, 7)

Page 28: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

28

3 7 5 1

8 4

2 6

Merge(A, 0, 0, 1)A:

Merge-Sort(A, 0, 7)

Page 29: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

29

3 7 5 1

8 42 6

Merge-Sort(A, 0, 1), returnA:

Merge-Sort(A, 0, 7)

Page 30: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

30

3 7 5 1

8 4

2 6

Merge-Sort(A, 2, 3)

48

, divideA:

Merge-Sort(A, 0, 7)

Page 31: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

31

3 7 5 1

4

2 6

8

Merge-Sort(A, 2, 2), base caseA:

Merge-Sort(A, 0, 7)

Page 32: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

32

3 7 5 1

4

2 6

8

Merge-Sort(A, 2, 2), returnA:

Merge-Sort(A, 0, 7)

Page 33: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

33

4

2 6

8

Merge-Sort(A, 3, 3), base caseA:

Merge-Sort(A, 0, 7)

Page 34: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

34

3 7 5 1

4

2 6

8

Merge-Sort(A, 3, 3), returnA:

Merge-Sort(A, 0, 7)

Page 35: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

35

3 7 5 1

2 6

4 8

Merge(A, 2, 2, 3)A:

Merge-Sort(A, 0, 7)

Page 36: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

36

3 7 5 1

2 6 4 8

Merge-Sort(A, 2, 3), returnA:

Merge-Sort(A, 0, 7)

Page 37: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

37

3 7 5 1

2 4 6 8

Merge(A, 0, 1, 3)A:

Merge-Sort(A, 0, 7)

Page 38: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

38

3 7 5 12 4 6 8Merge-Sort(A, 0, 3), return

A:

Merge-Sort(A, 0, 7)

Page 39: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

39

3 7 5 1

2 4 6 8Merge-Sort(A, 4, 7)

A:

Merge-Sort(A, 0, 7)

Page 40: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

40

1 3 5 7

2 4 6 8A:Merge (A, 4, 5, 7)

Merge-Sort(A, 0, 7)

Page 41: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

41

1 3 5 72 4 6 8Merge-Sort(A, 4, 7), return

A:

Merge-Sort(A, 0, 7)

Page 42: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

42

1 2 3 4 5 6 7 8Merge(A, 0, 3, 7)

A:

Merge-Sort(A, 0, 7)Merge-Sort(A, 0, 7), done!

Page 43: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

43

Merge-Sort Analysis cn

2 × cn/2 = cn

4 × cn/4 = cn

n/2 × 2c = cn

log n levels

• Total running time: (nlogn)• Total Space: (n)

Total: cn log n

n

n/2 n/2

n/4 n/4 n/4 n/4

2 2 2

Page 44: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

44

Merge-Sort SummaryApproach: divide and conquerTime

Most of the work is in the merging Total time: (n log n)

Space: (n), more space than other sorts.

Page 45: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

45

Overview Divide and Conquer

Merge Sort

Quick Sort

Page 46: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

46

Quick Sort Divide:

Pick any element p as the pivot, e.g, the first element

Partition the remaining elements into FirstPart, which contains all elements < pSecondPart, which contains all elements ≥ p

Recursively sort the FirstPart and SecondPart

Combine: no work is necessary since sorting is done in place

Page 47: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

47

Quick Sort

x < p p p ≤ x

PartitionFirstPart SecondPart

ppivot

A:

Recursive call

x < p p p ≤ x

SortedFirstPart

SortedSecondPart

SortedSorted

Page 48: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

48

Quick SortQuick-Sort(A, left, right)if left ≥ right return

else middle ← Partition(A, left,

right) Quick-Sort(A, left, middle–1 ) Quick-Sort(A, middle+1, right)

end if

Page 49: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

49

Partition

p

p x < p p ≤ x

p p ≤ xx < p

A:A:

A:A:

A:A:p

Page 50: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

50

Partition Example

A:A: 4 8 6 3 5 1 7 2

Page 51: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

51

Partition Example

A:A: 4 8 6 3 5 1 7 2

i=0i=0

j=1j=1

Page 52: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

52

Partition Example

A:A:

j=1j=1

4 8 6 3 5 1 7 2

i=0i=0

8

Page 53: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

53

Partition Example

A:A: 4 8 6 3 5 1 7 26

i=0i=0

j=2j=2

Page 54: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

54

Partition Example

A:A: 4 8 6 3 5 1 7 2

i=0i=0

383

j=3j=3

i=1i=1

Page 55: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

55

Partition Example

A:A: 4 3 6 8 5 1 7 2

i=1i=1

5

j=4j=4

Page 56: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

56

Partition Example

A:A: 4 3 6 8 5 1 7 2

i=1i=1

1

j=5j=5

Page 57: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

57

Partition Example

A:A: 4 3 6 8 5 1 7 2

i=2i=2

1 6

j=5j=5

Page 58: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

58

Partition Example

A:A: 4 3 8 5 7 2

i=2i=2

1 6 7

j=6j=6

Page 59: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

59

Partition Example

A:A: 4 3 8 5 7 2

i=2i=2

1 6 22 8

i=3i=3

j=7j=7

Page 60: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

60

Partition Example

A:A: 4 3 2 6 7 8

i=3i=3

1 5

j=8j=8

Page 61: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

61

Partition Example

A:A: 4 1 6 7 8

i=3i=3

2 542 3

Page 62: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

62

A:A: 3 6 7 81 542

x < 4x < 4 4 ≤ x4 ≤ x

pivot incorrect position

Partition Example

Page 63: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

63

Partition(A, left, right)1. x ← A[left]2. i ← left3. for j ← left+1 to right4. if A[j] < x then 5. i ← i + 16. swap(A[i], A[j])7. end if8. end for j9. swap(A[i], A[left])10. return i

n = right – left +1 Time: cn for some constant c Space: constant

Page 64: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

64

4 8 6 3 5 1 7 22 3 1 5 6 7 84

Quick-Sort(A, 0, 7)Partition

A:

Page 65: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

65

2 3 1

5 6 7 84

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 0, 2)

A:

, partition

Page 66: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

66

2

5 6 7 84

1

1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 0, 0) , base case, return

Page 67: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

67

2

5 6 7 84

1

33

Quick-Sort(A, 0, 7)Quick-Sort(A, 1, 1) , base case

Page 68: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

68

5 6 7 842 1 3

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 2, 2), returnQuick-Sort(A, 0, 2), return

Page 69: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

69

42 1 3

5 6 7 86 7 85

Quick-Sort(A, 0, 7)Quick-Sort(A, 2, 2), returnQuick-Sort(A, 4, 7) , partition

Page 70: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

70

4

5

6 7 87 866

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 5, 7) , partition

Page 71: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

71

4

5

6

7 887

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 6, 7) , partition

Page 72: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

72

4

5

6

7

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 7, 7)

8

, return, base case

8

Page 73: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

73

4

5

6 87

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 6, 7) , return

Page 74: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

74

4

5

2 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 5, 7) , return

6 87

Page 75: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

75

42 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 4, 7) , return

5 6 87

Page 76: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

76

42 1 3

Quick-Sort(A, 0, 7)Quick-Sort(A, 0, 7) , done!

5 6 87

Page 77: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

77

Quick-Sort: Best Case Even Partition

Total time: (nlogn)

cn

2 × cn/2 = cn

4 × c/4 = cn

n/3 × 3c = cn

log n levels

n

n/2 n/2

n/4

3 3 3

n/4n/4n/4

Page 78: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

78

cn

c(n-1)

3c

2c

n

n-1

n-2

3

2

c(n-2)

Happens only if input is sortd input is reversely sorted

Quick-Sort: Worst Case Unbalanced Partition

Total time: (n2)

Page 79: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

79

Quick-Sort: an Average Case Suppose the split is 1/10 : 9/10

Quick-Sort: an Average Case

cn

cn

cn

≤cn

n

0.1n 0.9n

0.01n 0.09n 0.09n

Total time: (nlogn)

0.81n

2

2

log10n

log10/9n

≤cn

Page 80: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

80

Quick-Sort Summary Time

Most of the work done in partitioning. Average case takes (n log(n)) time. Worst case takes (n2) time

Space Sorts in-place, i.e., does not require additional

space

Page 81: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

81

Summary Divide and Conquer

Merge-Sort Most of the work done in Merging (n log(n)) time (n) space

Quick-Sort Most of the work done in partitioning Average case takes (n log(n)) time Worst case takes (n2) time (1) space

Page 82: Sorting Algorithms Merge Sort Quick Sort Hairong Zhao  New Jersey Institute of Technology.

82

Homework1. What is the running time of Merge-Sort if the

array is already sorted? What is the best case running time of Merge-Sort?

2. Demonstrate the working of Partition on sequence (13, 5, 14, 11, 16, 12, 1, 15). What is the value of i returned at the completion of Partition?