Sorting Insertion Sort Merge Sort

download Sorting Insertion Sort Merge Sort

of 45

Transcript of Sorting Insertion Sort Merge Sort

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    1/45

    Sorting

    How to sort data efficiently?

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    2/45

    Sorting

    The sorting problem is to arrange a sequence of

    records so that the values of their key fields form

    a nondecreasing sequence.

    That is, given records r1, r2, . . . ,rn, with keyvalues k1, k2, . . . , kn, respectively, we must

    produce the same records in an order

    ri1, ri2 , . . . , rin

    such that ki1 ki2 . kin

    . The records all need not have distinct values

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    3/45

    Insertion Sort

    Insertion sort is a simple sorting algorithm

    that is appropriate for small inputs

    Its very time consuming so if the size of

    input is large, Insertion Sort is not a likely

    choice for sorting that data.

    Merge Sort, QuickSort are faster

    algorithms for sorting for that matter.

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    4/45

    Insertion Sort contin.

    The main idea of insertion sort is

    Start by considering the first two elements of the

    array data. If found out of order, swap them

    Consider the third element and insert it into itsproper position among the first three elements

    Consider the fourth element and place it in its

    proper position among the first four elementsand so on..

    After each ith iteration, first i elements should be

    sorted

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    5/45

    Insertion Sort (With an

    Example) In e game of cards, a player gets 13 cards.

    He keeps them in the sorted order in his hand for his

    ease.

    A player looks at the first two cards, sorts them andkeeps the smaller card first and then the second.

    Suppose that two cards were 9 and 8, the player swap

    them and keep 8 before 9.

    Now he takes the third card. Suppose, it is 10, then it isin its position. If this card is of number 2, the player will

    pick it up and put it on the start of the cards.

    Then he looks at the fourth card and inserts it in the first

    three cards (that he has sorted) at a proper place.

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    6/45

    Insertion (With an Example)

    cont. He repeats the same process with all the

    cards and finally gets the cards in a sorted

    order. Thus in this algorithm, we keep the

    left part of the array sorted and takeelement from the right and insert it in the

    left part at its proper place. Due to this

    process of insertion, it is called insertionsorting.

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    7/45

    Insertion Sort Cont

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    8/45

    Insertion Sort cont..

    The array consists of the elements 19, 12, 5 and 7.

    We take the first two numbers i.e. 19 and 12. As we see 12 is less

    than 19, so we swap their positions. Thus 12 comes at index 0 and

    19 goes to index 1.

    Now we pick the third number i.e. 5. We have to find the position ofthis number by comparing it with the two already sorted numbers.

    These numbers are 12 and 19. We see that 5 is smaller than these

    two. So it should come before these two numbers. Thus the proper

    position of 5 is index 0. To insert it at index 0, we shift the numbers

    12 and 19 before inserting 5 at index 0. Thus 5 has come at its

    position.

    Now we pick the number 7 and find its position between 5 and 12.

    To insert 7 after 5 and before 12, we have to shift the numbers 12

    and 19 to the right. After this shifting, we put number 7 at its

    position. Now the whole array has been sorted so the process stops

    here.

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    9/45

    Insertion Sort Cont.

    void insertionSort(int *arr, int N)

    {

    int pos, count, val;

    for(count=1; count < N; count++)

    {

    val = arr[count];

    for(pos=count-1; pos >= 0; pos--)

    if (arr[pos] > val)

    arr[pos+1]=arr[pos];else break;

    arr[pos+1] = val;

    }

    }

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    10/45

    Insertion Sort Analysis

    To insert the last element we need at most N-1 comparisons and N-

    1 movements.

    To insert the N-1stelement we need N-2 comparisons and N-2

    movements.

    .

    To insert the 2ndelement we need 1 comparison and one

    movement.

    To sum up:

    2* (1 + 2 + 3 + N - 1) = 2 * (N - 1)* N / 2 = (N-1)*N = (N2)

    If the greater part of the array is sorted, the complexity is almostO(N)

    The average complexity is proved to be = (N2)

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    11/45

    Insertion Sort an O(N^2)

    Algorithm Insertion Sort in an O(N^2) algorithm.

    Similarly, there are other O(N^2)

    algorithms for sorting like Selection Sort,

    Bubble Sort

    O(N^2) is not a very favorable amount of

    time we would like to spend on sorting.

    Lets try to understand O(nlogn) algorithms

    for sorting

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    12/45

    O(nlogn) Sorting Algorithms

    MergeSort

    Quick Sort

    Heap Sort

    Reading 11.1 (MergeSort), 11.2

    (QuickSort) from GTM

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    13/45

    Merge Sort

    MergeSort Algorithms fall under divide and

    conquercategory

    The divide and conquer strategy is well known in

    wars. The philosophy of this strategy is , divideyour enemy into parts and then conquer these

    parts. To conquer these parts is easy, as these

    parts cannot resist or react like a big united

    enemy. The same philosophy is applied in theabove algorithms. To understand the divide and

    conquer strategy in sorting algorithm, lets

    consider an example.

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    14/45

    Merge Sort Cont

    Divide and Conquer Example Suppose we have an unsorted array of

    numbers is given below.

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    15/45

    Merge Sort Cont

    Divide and Conquer Example Now we split this array into two parts

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    16/45

    Merge Sort Cont

    Divide and Conquer Example Now we have two parts of the array. We

    sort these parts separately. Suppose we

    sort these parts with an elementary sort

    algorithm. These parts may be sorted inthe following manner.

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    17/45

    Merge Sort Cont

    Divide and Conquer Example After this we merge these two parts and

    get the sorted array as shown below.

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    18/45

    Simple Analysis of Divide and

    Conquer Let see few analysis to confirm the usefulness

    of the divide and conquer technique.

    To sort the halves approximate time is

    (n/2)^2+(n/2)^2 To merge the two halves approximate time is n

    So, for n=100,

    divide and conquer takes approximately: =(100/2)^2 + (100/2)^2 + 100 = 2500 + 2500 +

    100 = 5100

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    19/45

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    20/45

    Divide and Conquer

    Advantages con. So the whole operation of sorting using this

    divide and conquer technique in insertion sort

    will take around

    (100/2)^2 + (100/2)^2+100 = 5100.Clearly the time spent (5100) after applying

    divide and conquer mechanism is significantly

    lesser than the previous time (10000). It is

    reduced approximately to half of the previous

    time. This example shows the usefulness of

    divide and conquer technique.

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    21/45

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    22/45

    Mergesort

    Merge-sort is based on an algorithmic design pattern called div ide-and-

    conquer .

    The divide-and-conquer pattern consists of the following three steps:

    1. Divide: If the in pu t size is smaller than a certain thresho ld (say, one

    or two elements), solve the problem directly using a straightforward method

    and return the solution obtained. Otherwise, divide the input data into two or more disjoint subsets.

    2. Recur: Recursively solve the subp roblems associated with the

    subsets.

    3. Conquer : Take the solut ions to the subprob lems and merge them

    into a solution to the original problem.

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    23/45

    Mergesort Con

    To sort a sequence S with n elements using the three divide-and-

    conquer

    steps, the merge-sort algorithm proceeds as follows:

    1. Divide: If S has zero or one element, return S immediately; it is

    alreadysorted. Otherwise (S has at least two elements), remove all the

    elements

    from S and put them into two sequences, S1 and S2, each

    containing about

    half of the elements of S; that is, S1 contains the first n/2elementsof S,

    and S2 contains the remaining n/2elements.

    2. Recur: Recursively sort sequences S1 and S2.

    3. Conquer : Put back the elements into S by merging the sorted

    sequences S1 and S2 into a sorted sequence.

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    24/45

    Mergesort Con

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    25/45

    Mergesort Con.

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    26/45

    Mergesort Con

    Merging Operation

    We have two sorted array and anotherempty array whose size is equal to the

    sum of sizes of two sorted arrays.

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    27/45

    Merging Operation Con.

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    28/45

    Merging Operation con

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    29/45

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    30/45

    Mergesort Con

    Divide and Recur Step

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    31/45

    Mergesort Con

    Divide and Recur Step

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    32/45

    Mergesort Con

    Divide and Recur Step

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    33/45

    Mergesort Con

    Divide and Recur Step

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    34/45

    Mergsort Con

    Divide and Recur Step

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    35/45

    Mergsort Con

    Divide and Recur Step

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    36/45

    Mergesort Con

    Divide and Recur Step

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    37/45

    Mergesort Con

    Food for Thought !

    Proposition

    The merge-sort tree associated with anexecution of mergesort on a sequence of

    size n has height log n. Why???

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    38/45

    The Running Time for

    Merging Let n1 and n2 be the number of elements of S1

    and S2, respectively.

    Algorithm merge has three while loops

    The key observation is that during eachiteration of one of the loops, one element is

    copied or moved from either S1 or S2 into S

    (and that element is no longer considered).

    Since no insertions are performed into S1 or S2,

    this observation implies that the overall number

    of iterations of the three loops is n1 +n2. Thus,

    the running time of algorithm merge is O(n1+n2).

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    39/45

    The Running Time of Merging

    con.

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    40/45

    Th R i Ti f M

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    41/45

    The Running Time of Merge-

    Sort con. let us analyze the running time of the

    entire merge-sort algorithm, assuming it is

    given an input sequence of n elements.

    For simplicity, we restrict our attention to

    the case where n is a power of 2.

    we analyze the merge-sort algorithm by

    referring to the merge-sort tree T. (RecallFigures 11.2 through 11.4.)

    Th R i Ti f M

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    42/45

    The Running Time of Merge-

    Sort cont We call the t ime spent at a node v o f T

    the runn ing t ime of the recu rsive cal l

    assoc iated w ith v, exc lud ing the time

    taken waiting for the recursive callsassociated with the children of v to

    terminate. In other words, the time spent

    at node v includes the running times of thedivide and conquer steps, but excludes the

    running time of the recur step.

    Th R i Ti f M

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    43/45

    The Running Time of Merge-

    Sort cont. the conquer step, which consists of

    merging two sorted subsequences, also

    takes linear time, independent of whether

    we are dealing with arrays or linked lists.That is, letting i denote the depth of node

    v, the time spent at node v is O(n/2^i),

    since the size of the sequence handled bythe recursive call associated with v is

    equal to n/2^i.

    Th R i Ti f M

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    44/45

    The Running Time of Merge-

    Sort con.. Given our definition of time spent at a node,

    the running time of merge-sort is equal to the

    sum of the times spent at the nodes of T.

    Observe that T has exactly 2^i nodes at depth i.This simple observation has an important

    consequence, for it implies that the overall time

    spent at all the nodes of T at depth i is O(2^i

    n/2^i), which is O(n). By Proposition 11.1(consult GTM Chapter 11, topic 11.1), the height

    of T is logn. Thus, since the time spent at each

    of the log n+1 levels of T is O(n), we have the

    following result.

    Th R i Ti f M

  • 8/12/2019 Sorting Insertion Sort Merge Sort

    45/45

    The Running Time of Merge-

    Sort con.Algorithm merge-sort sorts a sequence S

    of size n in O(nlogn) time, assuming two

    elements of S can be compared in O(1)

    time