Merge Sort Mit

8/8/2019 Merge Sort Mit

1/55

Introduction to Algorithms6.046J

Lecture 1Prof. Shafi Goldwasser

Prof. Erik Demaine


2/55

L1.2

Welcome toIntroduction to

Algorithms, Spring 2004

Handouts

1. Course Information

2. Course Calendar

3. Problem Set 1

4. Akra-Bazzi Handout
http://theory.lcs.mit.edu/classes/6.046/handouts/course-information.pdfhttp://theory.lcs.mit.edu/classes/6.046/handouts/calendar.pdfhttp://theory.lcs.mit.edu/classes/6.046/handouts/calendar.pdfhttp://theory.lcs.mit.edu/classes/6.046/handouts/course-information.pdf


3/55

L1.3

Course information

1. Staff

2. Prerequisites

3. Lectures & Recitations4. Handouts

5. Textbook (CLRS)

6. Website

8. Extra Help

9. Registration

10.Problem sets11.Describing algorithms

12.Grading policy

13.Collaboration policy
http://theory.lcs.mit.edu/classes/6.046/staff.htmlhttp://theory.lcs.mit.edu/classes/6.046/recitation.htmlhttp://theory.lcs.mit.edu/classes/6.046/handouts.htmlhttp://www.amazon.com/exec/obidos/ASIN/0262032937/qid=999563592/sr=1-3/ref=sc_b_3/002-9233126-5488026http://theory.lcs.mit.edu/classes/6.046/collaboration.htmlhttp://theory.lcs.mit.edu/classes/6.046/collaboration.htmlhttp://www.amazon.com/exec/obidos/ASIN/0262032937/qid=999563592/sr=1-3/ref=sc_b_3/002-9233126-5488026http://theory.lcs.mit.edu/classes/6.046/handouts.htmlhttp://theory.lcs.mit.edu/classes/6.046/recitation.htmlhttp://theory.lcs.mit.edu/classes/6.046/staff.html


4/55

L1.4

What is course about?

The theoretical study of design and

analysis of computer algorithms

Basic goals for an algorithm: always correct always terminates This class: performance Performance often draws the line betwee

what is possible and what is impossible.


5/55

L1.5

Design and Analysis of Algorithms

Analysis:predict the cost of an algorithm in

terms of resources and performance

Design: design algorithms which minimize the

cost


6/55

L1.6

Our Machine Model

Generic Random Access Machine (RAM)

Executes operations sequentially

Set of primitive operations: Arithmetic. Logical, Comparisons, Function calls

Simplifying assumption: all ops cost 1 unit

Eliminates dependence on the speed of our computer,otherwise impossible to verify and to compare


7/55L1.7

The problem of sorting

Input: sequence a1, a2, , an of numbers.

Example:

Input: 8 2 4 9 3 6

Output: 2 3 4 6 8 9

Output: permutation a'1, a'2, , a'n suchthat a'1 a'2 a'n.


8/55L1.8

Insertion sort

INSERTION-SORT (A, n) A[1 . . n]forj 2ton

do key A[j]

i j 1

while i > 0 andA[i] > keydo A[i+1] A[i]

i i 1

A[i+1] = key

pseudocode

i j

keysorted

A:1 n


9/55L1.9

Example of insertion sort

8 2 4 9 3 6


10/55L1.10


8 2 4 9 3 6


11/55L1.11


8 2 4 9 3 6

2 8 4 9 3 6


12/55L1.12


8 2 4 9 3 6

2 8 4 9 3 6


13/55L1.13


8 2 4 9 3 6

2 8 4 9 3 6

2 4 8 9 3 6


14/55

L1.14


8 2 4 9 3 6

2 8 4 9 3 6

2 4 8 9 3 6


15/55

L1.15


8 2 4 9 3 6

2 8 4 9 3 6

2 4 8 9 3 6

2 4 8 9 3 6


16/55

L1.16


8 2 4 9 3 6

2 8 4 9 3 6

2 4 8 9 3 6

2 4 8 9 3 6


17/55

L1.17


8 2 4 9 3 6

2 8 4 9 3 6

2 4 8 9 3 6

2 4 8 9 3 6

2 3 4 8 9 6


18/55

L1.18


8 2 4 9 3 6

2 8 4 9 3 6

2 4 8 9 3 6

2 4 8 9 3 6

2 3 4 8 9 6


19/55

L1.19


8 2 4 9 3 6

2 8 4 9 3 6

2 4 8 9 3 6

2 4 8 9 3 6

2 3 4 8 9 6

2 3 4 6 8 9 done


20/55

L1.20

Running time

The running time depends on the input: analready sorted sequence is easier to sort.

Major Simplifying Convention:Parameterize the running time by the size ofthe input, since short sequences are easier tosort than long ones.

TA(n) = time of A on length n inputs Generally, we seek upper bounds on the

running time, to have a guarantee of

performance.


21/55

L1.21

Kinds of analyses

Worst-case: (usually) T(n) = maximum time of algorithm

on any input of size n.

Average-case: (sometimes) T(n) = expected time of algorithmover all inputs of size n.

Need assumption of statistical

distribution of inputs.Best-case: (NEVER)

Cheat with a slow algorithm thatworks fast onsome input.


22/55

L1.22

Machine-independent time

What is insertion sorts worst-case time?

BIG IDEAS:

Ignore machine dependent constants,otherwise impossible to verify and to compare algorithms

Look atgrowthofT(n) as n .

Asymptotic AnalysisAsymptotic Analysis


23/55

L1.23

-notation

Drop low-order terms; ignore leading constants.

Example: 3n3 + 90n2 5n + 6046 = (n3)

DEF: (g(n)) = {f(n): there exist positive constants c1, c2, and

n0 such that 0

c1g(n)

f(n)

c2g(n)for all n n0}

Basic manipulations:


24/55

L1.24

Asymptotic performance

n

T(n)

n0

.

Asymptotic analysis is a

useful tool to help tostructure our thinkingtoward better algorithm

We shouldnt ignore

asymptoticallyslower algorithms,

however. Real-world design

situations often call for a

When n gets large enough, a (n2) algorithmalways beats a (n3) algorithm.


25/55

L1.25

Insertion sort analysis

Worst case: Input reverse sorted.

( )=

==n

j

njnT2

2)()(

Average case: All permutations equally likely.

( )=

==n

j

njnT2

2)2/()(

Is insertion sort a fast sorting algorithm? Moderately so, for small n.Not at all, for large n.

[arithmetic series]


26/55

L1.26

Example 2: Integer

Multiplication

Let X = A B and Y = C D where A,B,C

and D are n/2 bit integers

Simple Method: XY = (2n/2A+B)(2n/2C+D)

Running Time Recurrence

T(n) < 4T(n/2) + 100n

Solution T(n) = (n2)


27/55

L1.27

Better Integer Multiplication

Let X = A B and Y = C D where A,B,C and D

are n/2 bit integers

Karatsuba:XY = (2n/2+2n)AC+2n/2(A-B)(C-D) + (2n/2+1) BD

Running Time Recurrence

T(n) < 3T(n/2) + 100n

Solution: (n) = O(n log 3)


28/55

L1.28

Example 3:Merge sort

MERGE-SORTA[1 . . n]

1. Ifn = 1, done.

2. Recursively sortA[ 1 . . n/2 ] andA[ n/2 +1 . . n] .

3. Merge the 2 sorted lists.

Key subroutine:MERGE


29/55

L1.29

Merging two sorted arrays

20

13

7

2

12

11

9

1


30/55

L1.30


20

13

7

2

12

11

9

1

1


31/55

L1.31


20

13

7

2

12

11

9

1

1

20

13

7

2

12

11

9


32/55

L1.32


20

13

7

2

12

11

9

1

1

20

13

7

2

12

11

9

2


33/55

L1.33


20

13

7

2

12

11

9

1

1

20

13

7

2

12

11

9

2

20

13

7

12

11

9


34/55

L1.34


20

13

7

2

12

11

9

1

1

20

13

7

2

12

11

9

2

20

13

7

12

11

9

7


35/55

L1.35


20

13

7

2

12

11

9

1

1

20

13

7

2

12

11

9

2

20

13

7

12

11

9

7

20

13

12

11

9


36/55

L1.36


20

13

7

2

12

11

9

1

1

20

13

7

2

12

11

9

2

20

13

7

12

11

9

7

20

13

12

11

9

9


37/55

L1.37


20

13

7

2

12

11

9

1

1

20

13

7

2

12

11

9

2

20

13

7

12

11

9

7

20

13

12

11

9

9

20

13

12

11


38/55

L1.38


20

13

7

2

12

11

9

1

1

20

13

7

2

12

11

9

2

20

13

7

12

11

9

7

20

13

12

11

9

9

20

13

12

11

11


39/55

L1.39


20

13

7

2

12

11

9

1

1

20

13

7

2

12

11

9

2

20

13

7

12

11

9

7

20

13

12

11

9

9

20

13

12

11

11

20

13

12


40/55

L1.40


20

13

7

2

12

11

9

1

1

20

13

7

2

12

11

9

2

20

13

7

12

11

9

7

20

13

12

11

9

9

20

13

12

11

11

20

13

12

12


41/55

L1.41


20

13

7

2

12

11

9

1

1

20

13

7

2

12

11

9

2

20

13

7

12

11

9

7

20

13

12

11

9

9

20

13

12

11

11

20

13

12

12

Time = (n) to merge a totalofn elements (linear time).


42/55

L1.42

Analyzing merge sort

MERGE-SORTA[1 . . n]

1. Ifn = 1, done.

2. Recursively sortA[ 1 . . n/2 ] andA[ n/2 +1 . . n] .

3. Mergethe 2 sorted lists

T(n)

(1)

2T(n/2)

(n)

Sloppiness: Should be T( n/2 ) +T( n/2 ) , but it turns out not to matter

asymptotically.


43/55

L1.43

Recurrence for merge sort

T(n) = (1) ifn = 1;

2T(n/2) + (n) ifn > 1.

We shall usually omit stating the basecase when T(n) = (1) for sufficientlysmall n, but only when it has no effect onthe asymptotic solution to the recurrence.

Lecture 2 provides several ways to find agood upper bound on T(n).


44/55

L1.44

Recursion tree

Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.


45/55

L1.45

Recursion tree


T(n)


46/55

L1.46

Recursion tree


T(n/2) T(n/2)

cn


47/55

L1.47

Recursion tree


cn

T(n/4) T(n/4) T(n/4) T(n/4)

cn/2 cn/2


48/55

L1.48

Recursion tree


cn

cn/4 cn/4 cn/4 cn/4

cn/2 cn/2

(1)


49/55

L1.49

Recursion tree


cn

cn/4 cn/4 cn/4 cn/4

cn/2 cn/2

(1)

h = lg n


50/55

L1.50

Recursion tree


cn

cn/4 cn/4 cn/4 cn/4

cn/2 cn/2

(1)

h = lg n

cn


51/55

L1.51

Recursion tree


cn

cn/4 cn/4 cn/4 cn/4

cn/2 cn/2

(1)

h = lg n

cn

cn


52/55

L1.52

Recursion tree


cn

cn/4 cn/4 cn/4 cn/4

cn/2 cn/2

(1)

h = lg n

cn

cn

cn


53/55

L1.53

Recursion tree


cn

cn/4 cn/4 cn/4 cn/4

cn/2 cn/2

(1)

h = lg n

cn

cn

cn

#leaves = n (n)


54/55

L1.54

Recursion tree


cn

cn/4 cn/4 cn/4 cn/4

cn/2 cn/2

(1)

h = lg n

cn

cn

cn

#leaves = n (n)

Total = (n lg n)


55/55

Conclusions

(n lg n) grows more slowly than (n2).

Therefore, merge sort asymptoticallybeats insertion sort in the worst case.

In practice, merge sort beats insertionsort forn > 30 or so.

Merge Sort Mit

Documents

Transcript of Merge Sort Mit