• date post

03-Jun-2018
• Category

## Documents

• view

222

0

Embed Size (px)

### Transcript of Merge Sort Quick Sort

• 8/12/2019 Merge Sort Quick Sort

1/43

Analysis of Algorithms

CS 477/677

SortingPart B

Instructor: George Bebis

(Chapter 7)

• 8/12/2019 Merge Sort Quick Sort

2/43

2

Sorting

Insertion sort Design approach:

Sorts in place:

Best case:

Worst case:

Bubble Sort

Design approach:

Sorts in place:

Running time:

Yes

(n)

(n2)

incremental

Yes(n2)

incremental

• 8/12/2019 Merge Sort Quick Sort

3/43

3

Sorting

Selection sort Design approach:

Sorts in place:

Running time:

Merge Sort

Design approach:

Sorts in place:

Running time:

Yes

(n2)

incremental

No

Lets see!!

divide and conquer

• 8/12/2019 Merge Sort Quick Sort

4/43

4

Divide-and-Conquer

Dividethe problem into a number of sub-problems

Similar sub-problems of smaller size

Conquerthe sub-problems

Solve the sub-problems recursively

Sub-problem size small enough solve the problems in

straightforward manner

Combinethe solutions of the sub-problems

Obtain the solution for the original problem

• 8/12/2019 Merge Sort Quick Sort

5/43

5

Merge Sort Approach

To sort an array A[p . . r]:

Divide

Divide the n-element sequence to be sorted into two

subsequences of n/2elements each

Conquer

Sort the subsequences recursively using merge sort

When the size of the sequences is 1 there is nothing

more to do

Combine

Merge the two sorted subsequences

• 8/12/2019 Merge Sort Quick Sort

6/43

6

Merge Sort

Alg.:MERGE-SORT(A, p, r)

if p < r Check for base casethen q

(p + r)/2

Divide

MERGE-SORT(A, p, q) ConquerMERGE-SORT(A, q + 1, r) Conquer

MERGE(A, p, q, r)

Combine

Initial call:MERGE-SORT(A, 1, n)

1 2 3 4 5 6 7 8

62317425

p rq

• 8/12/2019 Merge Sort Quick Sort

7/43

7

ExamplenPower of 2

1 2 3 4 5 6 7 8

q = 462317425

1 2 3 4

7425

5 6 7 8

6231

1 2

25

3 4

74

5 6

31

7 8

62

1

5

2

2

3

4

4

7 1

6

3

7

2

8

6

5

Divide

• 8/12/2019 Merge Sort Quick Sort

8/43

8

ExamplenPower of 2

1

5

2

2

3

4

4

7 1

6

3

7

2

8

6

5

1 2 3 4 5 6 7 8

76543221

1 2 3 4

7542

5 6 7 8

6321

1 2

52

3 4

74

5 6

31

7 8

62

Conquer

and

Merge

• 8/12/2019 Merge Sort Quick Sort

9/43

9

ExamplenNot a Power of 2

62537416274

1 2 3 4 5 6 7 8 9 10 11

q = 6

416274

1 2 3 4 5 6

62537

7 8 9 10 11

q = 9q = 3

274

1 2 3

416

4 5 6

537

7 8 9

62

10 11

74

1 2

2

3

16

4 5

4

6

37

7 8

5

9

2

10

6

11

4

1

7

2

6

4

1

5

7

7

3

8

Divide

• 8/12/2019 Merge Sort Quick Sort

10/43

10

ExamplenNot a Power of 2

77665443221

1 2 3 4 5 6 7 8 9 10 11

764421

1 2 3 4 5 6

76532

7 8 9 10 11

742

1 2 3

641

4 5 6

753

7 8 9

62

10 11

2

3

4

6

5

9

2

10

6

11

4

1

7

2

6

4

1

5

7

7

3

8

74

1 2

61

4 5

73

7 8

Conquerand

Merge

• 8/12/2019 Merge Sort Quick Sort

11/43

11

Merging

Input:Array Aand indices p, q, rsuch thatp q < r

Subarrays A[p . . q]and A[q + 1 . . r]are sorted

Output: One single sorted subarray A[p . . r]

1 2 3 4 5 6 7 8

63217542

p rq

• 8/12/2019 Merge Sort Quick Sort

12/43

12

Merging

Idea for merging:

Two piles of sorted cards

Choose the smaller of the two top cards

Remove it and place it in the output pile

Repeat the process until one pile is empty

Take the remaining input pile and place it face-down

onto the output pile

1 2 3 4 5 6 7 8

63217542

p rq

A1A[p, q]

A2A[q+1, r]

A[p, r]

• 8/12/2019 Merge Sort Quick Sort

13/43

13

Example: MERGE(A, 9, 12, 16)

r

• 8/12/2019 Merge Sort Quick Sort

14/43

14

Example: MERGE(A, 9, 12, 16)

• 8/12/2019 Merge Sort Quick Sort

15/43

15

Example (cont.)

• 8/12/2019 Merge Sort Quick Sort

16/43

16

Example (cont.)

• 8/12/2019 Merge Sort Quick Sort

17/43

17

Example (cont.)

Done!

• 8/12/2019 Merge Sort Quick Sort

18/43

18

Merge - Pseudocode

Alg.:MERGE(A, p, q, r)1. Compute n1andn2

2. Copy the first n1elements intoL[1 . . n1+ 1]and the next n2elements into R[1 . . n2+ 1]

3. L[n1+ 1] ; R[n2+ 1] 4. i1; j1

5. for kpto r

6. do if L[ i ] R[ j ]

7. then A[k]L[ i ]8. ii + 1

9. else A[k]R[ j ]

10.j

j + 1

p q

7542

6321

rq + 1

L

R

1 2 3 4 5 6 7 8

63217542

p rq

n1 n2

R i Ti f M

• 8/12/2019 Merge Sort Quick Sort

19/43

19

Running Time of Merge

(assume last forloop)

Initialization (copying into temporary arrays): (n1+ n2) = (n)

Adding the elements to the final array:

- niterations, each taking constant time (n)

Total time for Merge:

(n)

• 8/12/2019 Merge Sort Quick Sort

20/43

20

Analyzing Divide-and Conquer Algorithms

The recurrence is based on the three steps of

T(n)running time on a problem of size n

Dividethe problem into asubproblems, each of size

n/b

:takes

D(n)

Conquer(solve) the subproblems aT(n/b)

Combinethe solutions C(n)

(1) if n cT(n) = aT(n/b) + D(n) + C(n) otherwise

• 8/12/2019 Merge Sort Quick Sort

21/43

21

MERGE-SORT Running Time

Divide: compute qas the average of pand r:D(n) = (1)

Conquer:

recursively solve 2 subproblems, each of size n/2

2T (n/2) Combine:

MERGE on an n-element subarray takes (n)time

C(n) =

(n)

(1) if n =1

T(n) = 2T(n/2) + (n) if n > 1

• 8/12/2019 Merge Sort Quick Sort

22/43

22

Solve the Recurrence

T(n) = c if n = 12T(n/2) + cn if n > 1

Use Masters Theorem:

Compare n with f(n) = cn

Case 2: T(n) = (nlgn)

• 8/12/2019 Merge Sort Quick Sort

23/43

23

Merge Sort - Discussion

Running time insensitive of the input

Guaranteed to run in (nlgn)

Requires extra space N

• 8/12/2019 Merge Sort Quick Sort

24/43

24

Sorting Challenge 1

Problem: Sort a file of huge records with tinykeys

Example application: Reorganize your MP-3 files

Which method to use?A. merge sort, guaranteed to run in time NlgN

B. selection sort

C. bubble sortD. a custom algorithm for huge records/tiny keys

E. insertion sort

S ti Fil ith H R d d

• 8/12/2019 Merge Sort Quick Sort

25/43

25

Sorting Files with Huge Records and

Small Keys

Insertion sort or bubble sort?

NO, too many exchanges

Selection sort?

YES, it takes lineartime for exchanges

Merge sort or custom method?

Probably not: selection sort simpler, does less swaps

• 8/12/2019 Merge Sort Quick Sort

26/43

26

Sorting Challenge 2

Problem: Sort a huge randomly-ordered file ofsmall records

Application: Process transaction record for a

phone company

Which sorting method to use?A. Bubble sort

B. Selection sortC. Mergesort guaranteed to run in time NlgN

D. Insertion sort

• 8/12/2019 Merge Sort Quick Sort

27/43

27

Sorting Huge, Randomly - Ordered Files

Selection sort? NO, always takes quadratic time

Bubble sort?

NO, quadratic time for randomly-ordered keys

Insertion sort?

NO, quadratic time for randomly-ordered keys

Mergesort?

YES, it is designed for this problem

• 8/12/2019 Merge Sort Quick Sort

28/43

28

Sorting Challenge 3

Problem: sort a file that is already almost inorder

Applications:

Re-sort a huge database after a few changes

Doublecheck that someone else sorted a file

Which sorting method to use?A. Mergesort, guaranteed to run in time NlgN

B. Selection sortC. Bubble sort

D. A custom algorithm for almost in-order files

E. Insertion sort

• 8/12/2019 Merge Sort Quick Sort

29/43

29

Sorting Files That are Almost in Order

Selection sort? NO, always takes quadratic time

Bubble sort?

NO, bad for some definitions of almost in order

Ex:B C D E F G H I J K L M N O P Q R S T U V W X Y Z A

Insertion sort?

YES, takes linear time for most definitions of almost

in order

Mergesort or custom method?

Probably not: insertion sort simpler and faster

• 8/12/2019 Merge Sort Quick Sort

30/43

30

Quicksort

Sort an array A[pr] Divide

Partition the array Ainto 2 subarrays A[p..q]and A[q+1..r], such

that each element of A[p..q]is smaller than or equal to each

element in A[q+1..r]

Need to find index qto partition the array

A[pq] A[q+1r]

• 8/12/2019 Merge Sort Quick Sort

31/43

31

Quicksort

Conquer

Recursively sort A[p..q]and A[q+1..r]using Quicksort

Combine

Trivial: the arrays are sorted in place

No additional work is required to combine them

The entire array is now sorted

A[pq] A[q+1r]

• 8/12/2019 Merge Sort Quick Sort

32/43

32

QUICKSORT

Alg.:QUICKSORT(A, p, r)

ifp < r

thenqPARTITION(A, p, r)

QUICKSORT (A, p, q)

QUICKSORT (A, q+1, r)

Recurrence:

Initially: p=1, r=n

PARTITION())T(n) = T(q) + T(nq) + f(n)

• 8/12/2019 Merge Sort Quick Sort

33/43

33

Partitioning the Array

Choosing PARTITION() There are different ways to do this

Hoare partition (see prob. 7-1, page 159)

Select a pivot element xaround which to partition

Grows two regions

A[pi] x

xA[jr]

A[pi] x x A[jr]

i j

• 8/12/2019 Merge Sort Quick Sort

34/43

34

Example

73146235

i j

75146233

i j

75146233

i j

75641233

i j

73146235

i j

A[pr]

75641233

ij

A[pq] A[q+1r]

pivot x=5

• 8/12/2019 Merge Sort Quick Sort

35/43

35

Example

• 8/12/2019 Merge Sort Quick Sort

36/43

36

Partitioning the Array

Alg.PARTITION (A, p, r)

1. x A[p]

2. i p 1

3. j r + 1

4. whileTRUE5. do repeatj j 1

6. untilA[j] x

7. do repeati i + 1

8. untilA[i] x9. ifi < j

10. thenexchange A[i] A[j]

11. elsereturnj

Running time: (n)n = r p + 1

73146235

i j

A:

arap

ij=q

A:

A[pq] A[q+1r]

p r

Each element is

visited once!

• 8/12/2019 Merge Sort Quick Sort

37/43

37

Recurrence

Alg.:QUICKSORT(A, p, r)

ifp < r

thenqPARTITION(A, p, r)

QUICKSORT (A, p, q)

QUICKSORT (A, q+1, r)

Recurrence:

Initially: p=1, r=n

T(n) = T(q) + T(nq) + n

• 8/12/2019 Merge Sort Quick Sort

38/43

38

Worst Case Partitioning

Worst-case partitioning One region has one element and the other has n1 elements

Maximally unbalanced

Recurrence: q=1

T(n) = T(1) + T(n1) + n,

T(1) = (1)

T(n) = T(n1) + n

=

2 2

1

1 ( ) ( ) ( )n

k

n k n n n

n

n - 1n - 2

n - 3

2

1

11

1

1

1

n

n

nn - 1

n - 2

3

2

(n2)

When does the worst case happen?

• 8/12/2019 Merge Sort Quick Sort

39/43

39

Best Case Partitioning

Best-case partitioning

Partitioning produces two regions of size n/2

Recurrence: q=n/2

T(n) = 2T(n/2) + (n)

T(n) =

(nlgn)(Master theorem)

• 8/12/2019 Merge Sort Quick Sort

40/43

40

Case Between Worst and Best

9-to-1 proportional splitQ(n) = Q(9n/10) + Q(n/10) + n

• 8/12/2019 Merge Sort Quick Sort

41/43

41

How does partition affect performance?

• 8/12/2019 Merge Sort Quick Sort

42/43

42

How does partition affect performance?

• 8/12/2019 Merge Sort Quick Sort

43/43

Performance of Quicksort

Average case

All permutations of the input numbers are equally likely

On a random input array, we will have a mixof well balanced

and unbalanced splits

Good and bad splits are randomly distributed across throughout

the tree

Alternate of a goodand a bad split

Nearly wellbalanced split

n

n - 11

(n 1)/2(n 1)/2

n

(n 1)/2(n 1)/2 + 1

Running time of Quicksort when levels alternatebetween good and bad splits is O(nlgn)

combined partitioning cost:

2n-1 = (n)

partitioning cost:

n = (n)