Intriduction to Algorithms (Lec1)

7/16/2019 Intriduction to Algorithms (Lec1)

http://slidepdf.com/reader/full/intriduction-to-algorithms-lec1 1/52

Introduction to Algorithms

6.046J/18.401J

LECTURE1

Analysis of Algorithms

•Insertion sort

• Asymptotic analysis

•Merge sort

•Recurrences

Copyright © 2001! Eri" #$ #emainean% Charles E$ Leiserson

Prof. Charles E. Leiserson



Course in&ormation

1$'ta&&

2$#istance learning

($)rere*uisites+$Lectures

!$Recitations

,$-an%outs.$Te/too"

$Course esite

3$E/tra help

10$Registration

11$)rolem sets

12$#escriing algorithms

1($4ra%ing policy1+$Collaoration policy

'eptemer .5 200! Intro%uction to Algorithms L1$2



Analysis o& algorithms

The theoretical study of computer-program

performance and resource usage.

6hat7s more important than per&ormance8•mo%ularity

•correctness

•maintainaility

•&unctionality

•roustness

•user&rien%liness

•programmer time

•simplicity•e/tensiility

•reliaility

'eptemer .5 200!Copyright © 2001! Eri" #$ #emaine an% Charles E$ Leiserson

Intro%uction to Algorithms L1$(



6hy stu%y algorithms an%

per&ormance8‧ Algorithms help us to un%erstan% scalaility$

‧)er&ormance o&ten %ras the line eteen hatis &easile an% hat is impossile$

‧ Algorithmic mathematics pro9i%es a language &or tal"ing aout program eha9ior$

‧)er&ormance is the currency o& computing$

‧The lessons o& program per&ormance generali:eto other computing resources$

‧'pee% is &un;


Intro%uction to Algorithms L1$+



The prolem o& sorting

Input < se*uence a⟨15 a25 =5 an⟩o& numers$

Output < permutation a>⟨ 15 a>25 =5 a>n ⟩'uch that

a>1?a>2?=?a>n$

E/ample<

Input < 2 + 3 ( ,

Output < 2 ( + , 3

'eptemer .5 200! L1$!Copyright © 2001! Eri" #$ #emaine an% Charles E$ Leiserson

Intro%uction to Algorithms



I@'ERTI@'RT

I@'ERTI@'RT B A5 n ⊳ AD1 $ $ n

&or j F2 to n

%o key F AD j

i F j G1

hile i H 0 an% ADi H key

%o ADi1 F ADi

i F i –1

ADi 1 J key

Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms L1$,'eptemer .5 200!

Kpseu%oco%e



Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms L1$.'eptemer .5 200!

I@'ERTI@'RT B A5 n ⊳ AD1 $ $ n

&or j F2 to n

%o key F AD j

i F j G1

hile i H 0 an% ADi H key

%o ADi1 F ADi

i F i –1

ADi 1 J key

Kpseu%oco%e

I@'ERTI@'RT

i

A<

sorte%key

n 1



E/ample o& insertion sort

2 + 3 ( ,

Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms L1$'eptemer .5 200!




2 + 3 ( ,

'eptemer .5 200! L1$3Copyright © 2001! Eri" #$ #emaine an% Charles E$ Leiserson




'eptemer .5 200! L1$10Copyright © 2001! Eri" #$ #emaine an% Charles E$ Leiserson


2 + 3 ( ,

2 + 3 ( ,





Intro%uction to Algorithms L1$11


2 + 3 ( ,

2 + 3 ( ,




Intro%uction to Algorithms L1$12


2 + 3 ( ,

2 + 3 ( ,

2 + 3 ( ,



Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$1(


2 + 3 ( ,

2 + 3 ( ,

2 + 3 ( ,




Intro%uction to Algorithms L1$1+


2 + 3 ( ,

2 + 3 ( ,

2 + 3 ( ,

2 + 3 ( ,



Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$1!


2 + 3 ( ,

2 + 3 ( ,

2 + 3 ( ,

2 + 3 ( ,



'eptemer .5 200! L1$1,Copyright © 2001! Eri" #$ #emaine an% Charles E$ Leiserson



2 + 3 ( ,

2 + 3 ( ,

2 + 3 ( ,

2 + 3 ( ,

2 ( + 3 ,



Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$1.

E/ample o& insertion sort 2 + 3 ( ,

2 + 3 ( ,

2 + 3 ( ,

2 + 3 ( ,

2 ( + 3 ,



Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$1

E/ample o& insertion sort 2 + 3 ( ,

2 + 3 ( ,

2 + 3 ( ,

2 + 3 ( ,

2 ( + 3 ,

2 ( + , 3 done



Running time

•The running time %epen%s on the input< an

alrea%y sorte% se*uence is easier to sort$

•)arameteri:e the running time y the si:e

o& the input5 since short se*uences are

easier to sort than long ones$

•4enerally5 e see" upper oun%s on the

running time5 ecause e9eryo%y li"es a

guarantee$




Nin%s o& analyses

6orstcase< Busually

• T Bn Jma/imum time o& algorithm on any

input o& si:e n$

A9eragecase< Bsometimes

• T Bn Je/pecte% time o& algorithm o9er all inputs

o& si:e n$

• @ee% assumption o& statistical %istriution o& inputs$

Oestcase< Bogus • Cheat ith a slo algorithm that or"s &ast on

some input$




KAsymptotic Analysis

Machinein%epen%ent time

What is insertion sort’s orst-case time!

•It %epen%s on the spee% o& our computer<

•relati9e spee% Bon the same machine5

•asolute spee% Bon %i&&erent machines$

OI4 I#EA<

•Ignore machine%epen%ent constants$

•Loo" at groth o& T Bn as nPQ$

KAsymptotic Analysis




notation

Math<

BgBn J S & Bn< there e/ist positi9e

constants c15 c25 an% n0

such that 0 ?c1gBn ?& Bn ?c2gBn

&or all nn0

Engineering<

•#rop loor%er termsV ignore lea%ing constants$

•E/ample< (n( 30n2 G!n ,0+, J Bn(




Asymptotic per&ormance

6hen n gets large enough5 a Bn2algorithm

alays eats a Bn(algorithm$

Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$2(

•6e shoul%n7t ignore

asymptotically sloeralgorithms5 hoe9er$

•Realorl% %esign

situations o&ten call &or

a care&ul alancing o&

engineering oecti9es$

•Asymptotic analysis is

a use&ul tool to help to

structure our thin"ing$

T"n#

nn$



Insertion sort analysis

Worst case% Input re9erse sorte%$

Darithmetic series

A&erage case% All permutations e*ually li"ely$

Is insertion sort a &ast sorting algorithm8

•Mo%erately so5 &or small n$

•@ot at all5 &or large n$

Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$2+

)()()( 2

2

n jnT n

j

Θ=Θ=∑=

)()2/()( 2

2

n jnT n

j

Θ=Θ=∑=



Merge sort

MER4E'RT AD1 $ $ n

1$I& nJ 15 %one$

2$Recursi9ely sort AD 1 $ $ $nW2$an% AD DnW21 $ $ n $

($KMerge the 2 sorte% lists$

Ney suroutine< MER4E

Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$2!



Merging to sorte% arrays

20 12

1( 11

. 3 2 1

Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$2,



Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$ 2.


20 12

1( 11

. 3 2 1

1





20 12

1( 11

. 3 2 1

1

20 12

1( 11

. 3 2





20 12

1( 11

. 3 2 1

1

20 12

1( 11

. 3 2

2



Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$(0


20 12

1( 11

. 3 2 1

1

20 12

1( 11

. 3 2

2

20 12

1( 11

. 3





20 12

1( 11

. 3 2 1

1

20 12

1( 11

. 3 2

20 12

1( 11

. 3

2 7





20 12

1( 11

. 3 2 1

1

20 12

1( 11

. 3 2

20 12

1( 11

. 3

27

20 12

1( 11

3



Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$((


20 12

1( 11

. 3 2 1

1

20 12

1( 11

. 3 2

20 12

1( 11

. 3

20 12

1( 11

3

2 7 9



Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$(+


20 12

1( 11

. 3 2 1

20 12

1( 11

. 3 2

20 12

1( 11

. 3

20 12

1( 11

3

20 12

1( 11

1 2 7 9



Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$(!


20 12

1( 11

. 3 2 1

20 12

1( 11

. 3 2

20 12

1( 11

. 3

20 12

1( 11

3

20 12

1( 11

1 2 7 9 11



Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$(,


20 12

1( 11

. 3

2 1

1

20 12

1( 11

. 3

2

2

20 12

1( 11

. 3

20 12

1( 11

3

20 12

1( 11

20 12

1(

7 9 11



Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$(.


20 12

1( 11

. 3

2 1

20 12

1( 11

. 3

2

20 12

1( 11

. 3

20 12

1( 11

3

20 12

1( 11

20 12

1(

1 2 7 9 11 12



Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$(.


20 12

1( 11

. 3

2 1

20 12

1( 11

. 3

2

20 12

1( 11

. 3

20 12

1( 11

3

20 12

1( 11

20 12

1(

1 2 7 9 11 12

Time J Bn to merge a total

o& n elements Blinear time$



Analy:ing merge sort

Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$(

MER4E'RTAD1 $ $ n

1$I& nJ 15 %one$

2$Recursi9ely sort AD 1 $ $ 「 nW2 」

an% AD 「nW2 」1 $ $ n $($KMergethe 2sorte% lists

'loppiness< 'houl% e TB 「

nW2 」

TB「

nW2」

5ut it turns out not to matter asymptotically$

TBn

B1

2TBnW2

Bn

Ause



Recurrence &or merge sort

• 6e shall usually omit stating the ase case

hen T"n# J B1 &or su&&iciently small n5 ut only

hen it has no e&&ect on the asymptotic solution

to the recurrence$• CLR' an% Lecture 2 pro9i%e se9eral ays to

&in% a goo% upper oun% on T Bn$


B1 i& nJ 1V

2T BnW2 Bn i& nH 1$T Bn J



Recursion tree


T Bn




Recursion tree


cn

T Bn'( T Bn'(




Recursion tree

'ol9e TBn J 2TBnW2 cn5 here c H 0 is constant$

cn

cnW2 cnW2

TBnW+ TBnW+ TBnW+ TBnW+

Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$+(



Recursion tree

'ol9e T Bn J 2T Bn'( cn5 here c H 0 is constant$cn

cnW2 cnW2

cnW+ cnW+ cnW+ cnW+

B1

Copyright © 2001! Eri" #$ #emaine an% Charles E$ LeisersonIntro%uction to Algorithms'eptemer .5 200! L1$++

．

．

．



Recursion tree


cnW2 cnW2

cnW+ cnW+ cnW+ cnW+

B1


．

．

．

hJ lgn



Recursion tree


cnW2 cnW2

cnW+ cnW+ cnW+ cnW+

B1


．

．

．

hJ lgn

cn



Recursion tree


cnW2 cnW2

cnW+ cnW+ cnW+ cnW+

B1



．

．

．

hJ lgn

cn

cn

cn

．

．

．



Recursion tree


cnW2 cnW2

cnW+ cnW+ cnW+ cnW+

B1



．

．

．

hJ lgn

cn

cn

cn

．

．

．

Xlea9es J n Bn



Recursion tree


cnW2 cnW2

cnW+ cnW+ cnW+ cnW+

B1



．

．

．

hJ lgn

cn

cn

cn

．

．

．

Xlea9es J n Bn

otal! "# n lg n$



Conclusions

• Bn lg n gros more sloly than Bn2$

• There&ore5 merge sort asymptotically eats

insertion sort in the orst case$

• In practice5 merge sort eats insertion sort

&or nH (0 or so$

• 4o test it out &or yoursel&;

Copyright © 2001 ! Eri" # #emaine an% Charles E Leiserson

Intriduction to Algorithms (Lec1)

Documents

Transcript of Intriduction to Algorithms (Lec1)