Merge Sort Mit

of 54

  • date post

    10-Apr-2018
  • Category

    Documents

  • view

    216
  • download

    0

Embed Size (px)

Transcript of Merge Sort Mit

  • 8/8/2019 Merge Sort Mit

    1/55

    Introduction to Algorithms6.046J

    Lecture 1Prof. Shafi Goldwasser

    Prof. Erik Demaine

  • 8/8/2019 Merge Sort Mit

    2/55

    L1.2

    Welcome toIntroduction to

    Algorithms, Spring 2004

    Handouts

    1. Course Information

    2. Course Calendar

    3. Problem Set 1

    4. Akra-Bazzi Handout

    http://theory.lcs.mit.edu/classes/6.046/handouts/course-information.pdfhttp://theory.lcs.mit.edu/classes/6.046/handouts/calendar.pdfhttp://theory.lcs.mit.edu/classes/6.046/handouts/calendar.pdfhttp://theory.lcs.mit.edu/classes/6.046/handouts/course-information.pdf
  • 8/8/2019 Merge Sort Mit

    3/55

    L1.3

    Course information

    1. Staff

    2. Prerequisites

    3. Lectures & Recitations4. Handouts

    5. Textbook (CLRS)

    6. Website

    8. Extra Help

    9. Registration

    10.Problem sets11.Describing algorithms

    12.Grading policy

    13.Collaboration policy

    http://theory.lcs.mit.edu/classes/6.046/staff.htmlhttp://theory.lcs.mit.edu/classes/6.046/recitation.htmlhttp://theory.lcs.mit.edu/classes/6.046/handouts.htmlhttp://www.amazon.com/exec/obidos/ASIN/0262032937/qid=999563592/sr=1-3/ref=sc_b_3/002-9233126-5488026http://theory.lcs.mit.edu/classes/6.046/collaboration.htmlhttp://theory.lcs.mit.edu/classes/6.046/collaboration.htmlhttp://www.amazon.com/exec/obidos/ASIN/0262032937/qid=999563592/sr=1-3/ref=sc_b_3/002-9233126-5488026http://theory.lcs.mit.edu/classes/6.046/handouts.htmlhttp://theory.lcs.mit.edu/classes/6.046/recitation.htmlhttp://theory.lcs.mit.edu/classes/6.046/staff.html
  • 8/8/2019 Merge Sort Mit

    4/55

    L1.4

    What is course about?

    The theoretical study of design and

    analysis of computer algorithms

    Basic goals for an algorithm: always correct always terminates This class: performance Performance often draws the line betwee

    what is possible and what is impossible.

  • 8/8/2019 Merge Sort Mit

    5/55

    L1.5

    Design and Analysis of Algorithms

    Analysis:predict the cost of an algorithm in

    terms of resources and performance

    Design: design algorithms which minimize the

    cost

  • 8/8/2019 Merge Sort Mit

    6/55

    L1.6

    Our Machine Model

    Generic Random Access Machine (RAM)

    Executes operations sequentially

    Set of primitive operations: Arithmetic. Logical, Comparisons, Function calls

    Simplifying assumption: all ops cost 1 unit

    Eliminates dependence on the speed of our computer,otherwise impossible to verify and to compare

  • 8/8/2019 Merge Sort Mit

    7/55L1.7

    The problem of sorting

    Input: sequence a1, a2, , an of numbers.

    Example:

    Input: 8 2 4 9 3 6

    Output: 2 3 4 6 8 9

    Output: permutation a'1, a'2, , a'n suchthat a'1 a'2 a'n.

  • 8/8/2019 Merge Sort Mit

    8/55L1.8

    Insertion sort

    INSERTION-SORT (A, n) A[1 . . n]forj 2ton

    do key A[j]

    i j 1

    while i > 0 andA[i] > keydo A[i+1] A[i]

    i i 1

    A[i+1] = key

    pseudocode

    i j

    keysorted

    A:1 n

  • 8/8/2019 Merge Sort Mit

    9/55L1.9

    Example of insertion sort

    8 2 4 9 3 6

  • 8/8/2019 Merge Sort Mit

    10/55L1.10

    Example of insertion sort

    8 2 4 9 3 6

  • 8/8/2019 Merge Sort Mit

    11/55L1.11

    Example of insertion sort

    8 2 4 9 3 6

    2 8 4 9 3 6

  • 8/8/2019 Merge Sort Mit

    12/55L1.12

    Example of insertion sort

    8 2 4 9 3 6

    2 8 4 9 3 6

  • 8/8/2019 Merge Sort Mit

    13/55L1.13

    Example of insertion sort

    8 2 4 9 3 6

    2 8 4 9 3 6

    2 4 8 9 3 6

  • 8/8/2019 Merge Sort Mit

    14/55

    L1.14

    Example of insertion sort

    8 2 4 9 3 6

    2 8 4 9 3 6

    2 4 8 9 3 6

  • 8/8/2019 Merge Sort Mit

    15/55

    L1.15

    Example of insertion sort

    8 2 4 9 3 6

    2 8 4 9 3 6

    2 4 8 9 3 6

    2 4 8 9 3 6

  • 8/8/2019 Merge Sort Mit

    16/55

    L1.16

    Example of insertion sort

    8 2 4 9 3 6

    2 8 4 9 3 6

    2 4 8 9 3 6

    2 4 8 9 3 6

  • 8/8/2019 Merge Sort Mit

    17/55

    L1.17

    Example of insertion sort

    8 2 4 9 3 6

    2 8 4 9 3 6

    2 4 8 9 3 6

    2 4 8 9 3 6

    2 3 4 8 9 6

  • 8/8/2019 Merge Sort Mit

    18/55

    L1.18

    Example of insertion sort

    8 2 4 9 3 6

    2 8 4 9 3 6

    2 4 8 9 3 6

    2 4 8 9 3 6

    2 3 4 8 9 6

  • 8/8/2019 Merge Sort Mit

    19/55

    L1.19

    Example of insertion sort

    8 2 4 9 3 6

    2 8 4 9 3 6

    2 4 8 9 3 6

    2 4 8 9 3 6

    2 3 4 8 9 6

    2 3 4 6 8 9 done

  • 8/8/2019 Merge Sort Mit

    20/55

    L1.20

    Running time

    The running time depends on the input: analready sorted sequence is easier to sort.

    Major Simplifying Convention:Parameterize the running time by the size ofthe input, since short sequences are easier tosort than long ones.

    TA(n) = time of A on length n inputs Generally, we seek upper bounds on the

    running time, to have a guarantee of

    performance.

  • 8/8/2019 Merge Sort Mit

    21/55

    L1.21

    Kinds of analyses

    Worst-case: (usually) T(n) = maximum time of algorithm

    on any input of size n.

    Average-case: (sometimes) T(n) = expected time of algorithmover all inputs of size n.

    Need assumption of statistical

    distribution of inputs.Best-case: (NEVER)

    Cheat with a slow algorithm thatworks fast onsome input.

  • 8/8/2019 Merge Sort Mit

    22/55

    L1.22

    Machine-independent time

    What is insertion sorts worst-case time?

    BIG IDEAS:

    Ignore machine dependent constants,otherwise impossible to verify and to compare algorithms

    Look atgrowthofT(n) as n .

    Asymptotic AnalysisAsymptotic Analysis

  • 8/8/2019 Merge Sort Mit

    23/55

    L1.23

    -notation

    Drop low-order terms; ignore leading constants.

    Example: 3n3 + 90n2 5n + 6046 = (n3)

    DEF: (g(n)) = {f(n): there exist positive constants c1, c2, and

    n0 such that 0

    c1g(n)

    f(n)

    c2g(n)for all n n0}

    Basic manipulations:

  • 8/8/2019 Merge Sort Mit

    24/55

    L1.24

    Asymptotic performance

    n

    T(n)

    n0

    .

    Asymptotic analysis is a

    useful tool to help tostructure our thinkingtoward better algorithm

    We shouldnt ignore

    asymptoticallyslower algorithms,

    however. Real-world design

    situations often call for a

    When n gets large enough, a (n2) algorithmalways beats a (n3) algorithm.

  • 8/8/2019 Merge Sort Mit

    25/55

    L1.25

    Insertion sort analysis

    Worst case: Input reverse sorted.

    ( )=

    ==n

    j

    njnT2

    2)()(

    Average case: All permutations equally likely.

    ( )=

    ==n

    j

    njnT2

    2)2/()(

    Is insertion sort a fast sorting algorithm? Moderately so, for small n.Not at all, for large n.

    [arithmetic series]

  • 8/8/2019 Merge Sort Mit

    26/55

    L1.26

    Example 2: Integer

    Multiplication

    Let X = A B and Y = C D where A,B,C

    and D are n/2 bit integers

    Simple Method: XY = (2n/2A+B)(2n/2C+D)

    Running Time Recurrence

    T(n) < 4T(n/2) + 100n

    Solution T(n) = (n2)

  • 8/8/2019 Merge Sort Mit

    27/55

    L1.27

    Better Integer Multiplication

    Let X = A B and Y = C D where A,B,C and D

    are n/2 bit integers

    Karatsuba:XY = (2n/2+2n)AC+2n/2(A-B)(C-D) + (2n/2+1) BD

    Running Time Recurrence

    T(n) < 3T(n/2) + 100n

    Solution: (n) = O(n log 3)

  • 8/8/2019 Merge Sort Mit

    28/55

    L1.28

    Example 3:Merge sort

    MERGE-SORTA[1 . . n]

    1. Ifn = 1, done.

    2. Recursively sortA[ 1 . . n/2 ] andA[ n/2 +1 . . n] .

    3. Merge the 2 sorted lists.

    Key subroutine:MERGE

  • 8/8/2019 Merge Sort Mit

    29/55

    L1.29

    Merging two sorted arrays

    20

    13

    7

    2

    12

    11

    9

    1

  • 8/8/2019 Merge Sort Mit

    30/55

    L1.30

    Merging two sorted arrays

    20

    13

    7

    2

    12

    11

    9

    1

    1

  • 8/8/2019 Merge Sort Mit

    31/55

    L1.31

    Merging two sorted arrays

    20

    13

    7

    2

    12

    11

    9

    1

    1

    20

    13

    7

    2

    12

    11

    9

  • 8/8/2019 Merge Sort Mit

    32/55

    L1.32

    Merging two sorted arrays

    20

    13

    7

    2

    12

    11

    9

    1

    1

    20

    13

    7

    2

    12

    11

    9

    2

  • 8/8/2019 Merge Sort Mit

    33/55

    L1.33

    Merging two sorted arrays

    20

    13

    7

    2

    12

    11

    9

    1

    1

    20

    13

    7

    2

    12

    11

    9

    2

    20

    13

    7

    12

    11

    9

  • 8/8/2019 Merge Sort Mit

    34/55

    L1.34

    Merging two sorted arrays

    20

    13

    7

    2

    12

    11

    9

    1

    1

    20

    13

    7

    2

    12

    11

    9

    2

    20

    13

    7

    12

    11

    9

    7

  • 8/8/2019 Merge Sort Mit

    35/55

    L1.35

    Merging two sorted arrays

    20

    13

    7

    2

    12

    11

    9

    1

    1

    20

    13

    7

    2

    12

    11

    9

    2

    20

    13

    7

    12

    11

    9

    7

    20

    13

    12

    11

    9

  • 8/8/2019 Merge Sort Mit

    36/55

    L1.36

    Merging two sorted arrays

    20

    13

    7

    2

    12

    11

    9

    1

    1

    20

    13

    7

    2

    12

    11

    9

    2

    20

    13

    7

    12

    11

    9

    7

    20

    13

    12

    11

    9

    9

  • 8/8/2019 Merge Sort Mit

    37/55

    L1.37

    Merging two sorted arrays

    20

    13

    7

    2

    12

    11

    9

    1

    1

    20

    13

    7

    2

    12

    11

    9

    2

    20

    13

    7

    12

    11

    9

    7

    20

    13

    12

    11

    9

    9

    20

    13

    12

    11

  • 8/8/2019 Merge Sort Mit

    38/55

    L1.38

    Merging two sorted arrays

    20

    13

    7

    2

    12

    11

    9

    1

    1

    20

    13

    7

    2

    12

    11

    9

    2

    20

    13

    7

    12

    11

    9

    7

    20

    13

    12

    11

    9

    9

    20

    13

    12

    11

    11

  • 8/8/2019 Merge Sort Mit

    39/55

    L1.39

    Merging two sorted arrays

    20

    13

    7

    2

    12

    11

    9

    1

    1

    20

    13

    7

    2

    12

    11

    9

    2

    20

    13

    7

    12

    11

    9

    7

    20

    13

    12

    11

    9

    9

    20

    13

    12

    11

    11

    20

    13

    12

  • 8/8/2019 Merge Sort Mit

    40/55

    L1.40

    Merging two sorted arrays

    20

    13

    7

    2

    12

    11

    9

    1

    1

    20

    13

    7

    2

    12

    11

    9

    2

    20

    13

    7

    12

    11

    9

    7

    20

    13

    12

    11

    9

    9

    20

    13

    12

    11

    11

    20

    13

    12

    12

  • 8/8/2019 Merge Sort Mit

    41/55

    L1.41

    Merging two sorted arrays

    20

    13

    7

    2

    12

    11

    9

    1

    1

    20

    13

    7

    2

    12

    11

    9

    2

    20

    13

    7

    12

    11

    9

    7

    20

    13

    12

    11

    9

    9

    20

    13

    12

    11

    11

    20

    13

    12

    12

    Time = (n) to merge a totalofn elements (linear time).

  • 8/8/2019 Merge Sort Mit

    42/55

    L1.42

    Analyzing merge sort

    MERGE-SORTA[1 . . n]

    1. Ifn = 1, done.

    2. Recursively sortA[ 1 . . n/2 ] andA[ n/2 +1 . . n] .

    3. Mergethe 2 sorted lists

    T(n)

    (1)

    2T(n/2)

    (n)

    Sloppiness: Should be T( n/2 ) +T( n/2 ) , but it turns out not to matter

    asymptotically.

  • 8/8/2019 Merge Sort Mit

    43/55

    L1.43

    Recurrence for merge sort

    T(n) = (1) ifn = 1;

    2T(n/2) + (n) ifn > 1.

    We shall usually omit stating the basecase when T(n) = (1) for sufficientlysmall n, but only when it has no effect onthe asymptotic solution to the recurrence.

    Lecture 2 provides several ways to find agood upper bound on T(n).

  • 8/8/2019 Merge Sort Mit

    44/55

    L1.44

    Recursion tree

    Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

  • 8/8/2019 Merge Sort Mit

    45/55

    L1.45

    Recursion tree

    Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

    T(n)

  • 8/8/2019 Merge Sort Mit

    46/55

    L1.46

    Recursion tree

    Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

    T(n/2) T(n/2)

    cn

  • 8/8/2019 Merge Sort Mit

    47/55

    L1.47

    Recursion tree

    Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

    cn

    T(n/4) T(n/4) T(n/4) T(n/4)

    cn/2 cn/2

  • 8/8/2019 Merge Sort Mit

    48/55

    L1.48

    Recursion tree

    Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

    cn

    cn/4 cn/4 cn/4 cn/4

    cn/2 cn/2

    (1)

  • 8/8/2019 Merge Sort Mit

    49/55

    L1.49

    Recursion tree

    Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

    cn

    cn/4 cn/4 cn/4 cn/4

    cn/2 cn/2

    (1)

    h = lg n

  • 8/8/2019 Merge Sort Mit

    50/55

    L1.50

    Recursion tree

    Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

    cn

    cn/4 cn/4 cn/4 cn/4

    cn/2 cn/2

    (1)

    h = lg n

    cn

  • 8/8/2019 Merge Sort Mit

    51/55

    L1.51

    Recursion tree

    Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

    cn

    cn/4 cn/4 cn/4 cn/4

    cn/2 cn/2

    (1)

    h = lg n

    cn

    cn

  • 8/8/2019 Merge Sort Mit

    52/55

    L1.52

    Recursion tree

    Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

    cn

    cn/4 cn/4 cn/4 cn/4

    cn/2 cn/2

    (1)

    h = lg n

    cn

    cn

    cn

  • 8/8/2019 Merge Sort Mit

    53/55

    L1.53

    Recursion tree

    Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

    cn

    cn/4 cn/4 cn/4 cn/4

    cn/2 cn/2

    (1)

    h = lg n

    cn

    cn

    cn

    #leaves = n (n)

  • 8/8/2019 Merge Sort Mit

    54/55

    L1.54

    Recursion tree

    Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

    cn

    cn/4 cn/4 cn/4 cn/4

    cn/2 cn/2

    (1)

    h = lg n

    cn

    cn

    cn

    #leaves = n (n)

    Total = (n lg n)

  • 8/8/2019 Merge Sort Mit

    55/55

    Conclusions

    (n lg n) grows more slowly than (n2).

    Therefore, merge sort asymptoticallybeats insertion sort in the worst case.

    In practice, merge sort beats insertionsort forn > 30 or so.