Merge Sort Quick Sort

download Merge Sort Quick Sort

of 43

  • date post

    03-Jun-2018
  • Category

    Documents

  • view

    222
  • download

    0

Embed Size (px)

Transcript of Merge Sort Quick Sort

  • 8/12/2019 Merge Sort Quick Sort

    1/43

    Analysis of Algorithms

    CS 477/677

    SortingPart B

    Instructor: George Bebis

    (Chapter 7)

  • 8/12/2019 Merge Sort Quick Sort

    2/43

    2

    Sorting

    Insertion sort Design approach:

    Sorts in place:

    Best case:

    Worst case:

    Bubble Sort

    Design approach:

    Sorts in place:

    Running time:

    Yes

    (n)

    (n2)

    incremental

    Yes(n2)

    incremental

  • 8/12/2019 Merge Sort Quick Sort

    3/43

    3

    Sorting

    Selection sort Design approach:

    Sorts in place:

    Running time:

    Merge Sort

    Design approach:

    Sorts in place:

    Running time:

    Yes

    (n2)

    incremental

    No

    Lets see!!

    divide and conquer

  • 8/12/2019 Merge Sort Quick Sort

    4/43

    4

    Divide-and-Conquer

    Dividethe problem into a number of sub-problems

    Similar sub-problems of smaller size

    Conquerthe sub-problems

    Solve the sub-problems recursively

    Sub-problem size small enough solve the problems in

    straightforward manner

    Combinethe solutions of the sub-problems

    Obtain the solution for the original problem

  • 8/12/2019 Merge Sort Quick Sort

    5/43

    5

    Merge Sort Approach

    To sort an array A[p . . r]:

    Divide

    Divide the n-element sequence to be sorted into two

    subsequences of n/2elements each

    Conquer

    Sort the subsequences recursively using merge sort

    When the size of the sequences is 1 there is nothing

    more to do

    Combine

    Merge the two sorted subsequences

  • 8/12/2019 Merge Sort Quick Sort

    6/43

    6

    Merge Sort

    Alg.:MERGE-SORT(A, p, r)

    if p < r Check for base casethen q

    (p + r)/2

    Divide

    MERGE-SORT(A, p, q) ConquerMERGE-SORT(A, q + 1, r) Conquer

    MERGE(A, p, q, r)

    Combine

    Initial call:MERGE-SORT(A, 1, n)

    1 2 3 4 5 6 7 8

    62317425

    p rq

  • 8/12/2019 Merge Sort Quick Sort

    7/43

    7

    ExamplenPower of 2

    1 2 3 4 5 6 7 8

    q = 462317425

    1 2 3 4

    7425

    5 6 7 8

    6231

    1 2

    25

    3 4

    74

    5 6

    31

    7 8

    62

    1

    5

    2

    2

    3

    4

    4

    7 1

    6

    3

    7

    2

    8

    6

    5

    Divide

  • 8/12/2019 Merge Sort Quick Sort

    8/43

    8

    ExamplenPower of 2

    1

    5

    2

    2

    3

    4

    4

    7 1

    6

    3

    7

    2

    8

    6

    5

    1 2 3 4 5 6 7 8

    76543221

    1 2 3 4

    7542

    5 6 7 8

    6321

    1 2

    52

    3 4

    74

    5 6

    31

    7 8

    62

    Conquer

    and

    Merge

  • 8/12/2019 Merge Sort Quick Sort

    9/43

    9

    ExamplenNot a Power of 2

    62537416274

    1 2 3 4 5 6 7 8 9 10 11

    q = 6

    416274

    1 2 3 4 5 6

    62537

    7 8 9 10 11

    q = 9q = 3

    274

    1 2 3

    416

    4 5 6

    537

    7 8 9

    62

    10 11

    74

    1 2

    2

    3

    16

    4 5

    4

    6

    37

    7 8

    5

    9

    2

    10

    6

    11

    4

    1

    7

    2

    6

    4

    1

    5

    7

    7

    3

    8

    Divide

  • 8/12/2019 Merge Sort Quick Sort

    10/43

    10

    ExamplenNot a Power of 2

    77665443221

    1 2 3 4 5 6 7 8 9 10 11

    764421

    1 2 3 4 5 6

    76532

    7 8 9 10 11

    742

    1 2 3

    641

    4 5 6

    753

    7 8 9

    62

    10 11

    2

    3

    4

    6

    5

    9

    2

    10

    6

    11

    4

    1

    7

    2

    6

    4

    1

    5

    7

    7

    3

    8

    74

    1 2

    61

    4 5

    73

    7 8

    Conquerand

    Merge

  • 8/12/2019 Merge Sort Quick Sort

    11/43

    11

    Merging

    Input:Array Aand indices p, q, rsuch thatp q < r

    Subarrays A[p . . q]and A[q + 1 . . r]are sorted

    Output: One single sorted subarray A[p . . r]

    1 2 3 4 5 6 7 8

    63217542

    p rq

  • 8/12/2019 Merge Sort Quick Sort

    12/43

    12

    Merging

    Idea for merging:

    Two piles of sorted cards

    Choose the smaller of the two top cards

    Remove it and place it in the output pile

    Repeat the process until one pile is empty

    Take the remaining input pile and place it face-down

    onto the output pile

    1 2 3 4 5 6 7 8

    63217542

    p rq

    A1A[p, q]

    A2A[q+1, r]

    A[p, r]

  • 8/12/2019 Merge Sort Quick Sort

    13/43

    13

    Example: MERGE(A, 9, 12, 16)

    r

  • 8/12/2019 Merge Sort Quick Sort

    14/43

    14

    Example: MERGE(A, 9, 12, 16)

  • 8/12/2019 Merge Sort Quick Sort

    15/43

    15

    Example (cont.)

  • 8/12/2019 Merge Sort Quick Sort

    16/43

    16

    Example (cont.)

  • 8/12/2019 Merge Sort Quick Sort

    17/43

    17

    Example (cont.)

    Done!

  • 8/12/2019 Merge Sort Quick Sort

    18/43

    18

    Merge - Pseudocode

    Alg.:MERGE(A, p, q, r)1. Compute n1andn2

    2. Copy the first n1elements intoL[1 . . n1+ 1]and the next n2elements into R[1 . . n2+ 1]

    3. L[n1+ 1] ; R[n2+ 1] 4. i1; j1

    5. for kpto r

    6. do if L[ i ] R[ j ]

    7. then A[k]L[ i ]8. ii + 1

    9. else A[k]R[ j ]

    10.j

    j + 1

    p q

    7542

    6321

    rq + 1

    L

    R

    1 2 3 4 5 6 7 8

    63217542

    p rq

    n1 n2

    R i Ti f M

  • 8/12/2019 Merge Sort Quick Sort

    19/43

    19

    Running Time of Merge

    (assume last forloop)

    Initialization (copying into temporary arrays): (n1+ n2) = (n)

    Adding the elements to the final array:

    - niterations, each taking constant time (n)

    Total time for Merge:

    (n)

  • 8/12/2019 Merge Sort Quick Sort

    20/43

    20

    Analyzing Divide-and Conquer Algorithms

    The recurrence is based on the three steps of

    the paradigm:

    T(n)running time on a problem of size n

    Dividethe problem into asubproblems, each of size

    n/b

    :takes

    D(n)

    Conquer(solve) the subproblems aT(n/b)

    Combinethe solutions C(n)

    (1) if n cT(n) = aT(n/b) + D(n) + C(n) otherwise

  • 8/12/2019 Merge Sort Quick Sort

    21/43

    21

    MERGE-SORT Running Time

    Divide: compute qas the average of pand r:D(n) = (1)

    Conquer:

    recursively solve 2 subproblems, each of size n/2

    2T (n/2) Combine:

    MERGE on an n-element subarray takes (n)time

    C(n) =

    (n)

    (1) if n =1

    T(n) = 2T(n/2) + (n) if n > 1

  • 8/12/2019 Merge Sort Quick Sort

    22/43

    22

    Solve the Recurrence

    T(n) = c if n = 12T(n/2) + cn if n > 1

    Use Masters Theorem:

    Compare n with f(n) = cn

    Case 2: T(n) = (nlgn)

  • 8/12/2019 Merge Sort Quick Sort

    23/43

    23

    Merge Sort - Discussion

    Running time insensitive of the input

    Advantages:

    Guaranteed to run in (nlgn)

    Disadvantage

    Requires extra space N

  • 8/12/2019 Merge Sort Quick Sort

    24/43

    24

    Sorting Challenge 1

    Problem: Sort a file of huge records with tinykeys

    Example application: Reorganize your MP-3 files

    Which method to use?A. merge sort, guaranteed to run in time NlgN

    B. selection sort

    C. bubble sortD. a custom algorithm for huge records/tiny keys

    E. insertion sort

    S ti Fil ith H R d d

  • 8/12/2019 Merge Sort Quick Sort

    25/43

    25

    Sorting Files with Huge Records and

    Small Keys

    Insertion sort or bubble sort?

    NO, too many exchanges

    Selection sort?

    YES, it takes lineartime for exchanges

    Merge sort or custom method?

    Probably not: selection sort simpler, does less swaps

  • 8/12/2019 Merge Sort Quick Sort

    26/43

    26

    Sorting Challenge 2

    Problem: Sort a huge randomly-ordered file ofsmall records

    Application: Process transaction record for a

    phone company

    Which sorting method to use?A. Bubble sort

    B. Selection sortC. Mergesort guaranteed to run in time NlgN

    D. Insertion sort

  • 8/12/2019 Merge Sort Quick Sort

    27/43

    27

    Sorting Huge, Randomly - Ordered Files

    Selection sort? NO, always takes quadratic time

    Bubble sort?

    NO, quadratic time for randomly-ordered keys

    Insertion sort?

    NO, quadratic time for randomly-ordered keys

    Mergesort?

    YES, it is designed for this problem

  • 8/12/2019 Merge Sort Quick Sort

    28/43

    28

    Sorting Challenge 3

    Problem: sort a file that is already almost inorder

    Applications:

    Re-sort a huge database after a few changes

    Doublecheck that someone else sorted a file

    Which sorting method to use?A. Mergesort, guaranteed to run in time NlgN

    B. Selection sortC. Bubble sort

    D. A custom algorithm for almost in-order files

    E. Insertion sort

  • 8/12/2019 Merge Sort Quick Sort

    29/43

    29

    Sorting Files That are Almost in Order

    Selection sort? NO, always takes quadratic time

    Bubble sort?

    NO, bad for some definitions of almost in order

    Ex:B C D E F G H I J K L M N O P Q R S T U V W X Y Z A

    Insertion sort?

    YES, takes linear time for most definitions of almost

    in order

    Mergesort or custom method?

    Probably not: insertion sort simpler and faster

  • 8/12/2019 Merge Sort Quick Sort

    30/43

    30

    Quicksort

    Sort an array A[pr] Divide

    Partition the array Ainto 2 subarrays A[p..q]and A[q+1..r], such

    that each element of A[p..q]is smaller than or equal to each

    element in A[q+1..r]

    Need to find index qto partition the array

    A[pq] A[q+1r]

  • 8/12/2019 Merge Sort Quick Sort

    31/43

    31

    Quicksort

    Conquer

    Recursively sort A[p..q]and A[q+1..r]using Quicksort

    Combine

    Trivial: the arrays are sorted in place

    No additional work is required to combine them

    The entire array is now sorted

    A[pq] A[q+1r]

  • 8/12/2019 Merge Sort Quick Sort

    32/43

    32

    QUICKSORT

    Alg.:QUICKSORT(A, p, r)

    ifp < r

    thenqPARTITION(A, p, r)

    QUICKSORT (A, p, q)

    QUICKSORT (A, q+1, r)

    Recurrence:

    Initially: p=1, r=n

    PARTITION())T(n) = T(q) + T(nq) + f(n)

  • 8/12/2019 Merge Sort Quick Sort

    33/43

    33

    Partitioning the Array

    Choosing PARTITION() There are different ways to do this

    Each has its own advantages/disadvantages

    Hoare partition (see prob. 7-1, page 159)

    Select a pivot element xaround which to partition

    Grows two regions

    A[pi] x

    xA[jr]

    A[pi] x x A[jr]

    i j

  • 8/12/2019 Merge Sort Quick Sort

    34/43

    34

    Example

    73146235

    i j

    75146233

    i j

    75146233

    i j

    75641233

    i j

    73146235

    i j

    A[pr]

    75641233

    ij

    A[pq] A[q+1r]

    pivot x=5

  • 8/12/2019 Merge Sort Quick Sort

    35/43

    35

    Example

  • 8/12/2019 Merge Sort Quick Sort

    36/43

    36

    Partitioning the Array

    Alg.PARTITION (A, p, r)

    1. x A[p]

    2. i p 1

    3. j r + 1

    4. whileTRUE5. do repeatj j 1

    6. untilA[j] x

    7. do repeati i + 1

    8. untilA[i] x9. ifi < j

    10. thenexchange A[i] A[j]

    11. elsereturnj

    Running time: (n)n = r p + 1

    73146235

    i j

    A:

    arap

    ij=q

    A:

    A[pq] A[q+1r]

    p r

    Each element is

    visited once!

  • 8/12/2019 Merge Sort Quick Sort

    37/43

    37

    Recurrence

    Alg.:QUICKSORT(A, p, r)

    ifp < r

    thenqPARTITION(A, p, r)

    QUICKSORT (A, p, q)

    QUICKSORT (A, q+1, r)

    Recurrence:

    Initially: p=1, r=n

    T(n) = T(q) + T(nq) + n

  • 8/12/2019 Merge Sort Quick Sort

    38/43

    38

    Worst Case Partitioning

    Worst-case partitioning One region has one element and the other has n1 elements

    Maximally unbalanced

    Recurrence: q=1

    T(n) = T(1) + T(n1) + n,

    T(1) = (1)

    T(n) = T(n1) + n

    =

    2 2

    1

    1 ( ) ( ) ( )n

    k

    n k n n n

    n

    n - 1n - 2

    n - 3

    2

    1

    11

    1

    1

    1

    n

    n

    nn - 1

    n - 2

    3

    2

    (n2)

    When does the worst case happen?

  • 8/12/2019 Merge Sort Quick Sort

    39/43

    39

    Best Case Partitioning

    Best-case partitioning

    Partitioning produces two regions of size n/2

    Recurrence: q=n/2

    T(n) = 2T(n/2) + (n)

    T(n) =

    (nlgn)(Master theorem)

  • 8/12/2019 Merge Sort Quick Sort

    40/43

    40

    Case Between Worst and Best

    9-to-1 proportional splitQ(n) = Q(9n/10) + Q(n/10) + n

  • 8/12/2019 Merge Sort Quick Sort

    41/43

    41

    How does partition affect performance?

  • 8/12/2019 Merge Sort Quick Sort

    42/43

    42

    How does partition affect performance?

  • 8/12/2019 Merge Sort Quick Sort

    43/43

    Performance of Quicksort

    Average case

    All permutations of the input numbers are equally likely

    On a random input array, we will have a mixof well balanced

    and unbalanced splits

    Good and bad splits are randomly distributed across throughout

    the tree

    Alternate of a goodand a bad split

    Nearly wellbalanced split

    n

    n - 11

    (n 1)/2(n 1)/2

    n

    (n 1)/2(n 1)/2 + 1

    Running time of Quicksort when levels alternatebetween good and bad splits is O(nlgn)

    combined partitioning cost:

    2n-1 = (n)

    partitioning cost:

    n = (n)