Introduction to Algorithms Lecture 1. Introduction The methods of algorithm design form one of the...

Introduction to AlgorithmsIntroduction to Algorithms

Lecture 1

IntroductionIntroduction

• The methods of algorithm design form one of the core practical technologies of computer science.

• The main aim of this lecture is to familiarize the student with the framework we shall use through the course about the design and analysis of algorithms.

• We start with a discussion of the algorithms needed to

solve computational problems. The problem of sorting is used as a running example.

• We introduce a pseudocode to show how we shall

specify the algorithms.

AlgorithmsAlgorithms

• The word algorithm comes from the name of a Persian mathematician Abu Ja’far Mohammed ibn-i Musa al Khowarizmi.

• In computer science, this word refers to a special method useable by a computer for solution of a problem. The statement of the problem specifies in general terms the desired input/output relationship.

• For example, sorting a given sequence of numbers into nondecreasing order provides fertile ground for introducing many standard design techniques and analysis tools.

The problem of sortingThe problem of sorting

Insertion SortInsertion Sort

Example of Insertion SortExample of Insertion Sort

Analysis of algorithmsAnalysis of algorithms

The theoretical study of computer-programperformance and resource usage.

What’s more important than performance?• modularity

• correctness• maintainability• functionality• robustness

• user-friendliness• programmer time

• simplicity• extensibility

• reliability

Analysis of algorithmsAnalysis of algorithms

Why study algorithms and performance?

• Algorithms help us to understand scalability.

• Performance often draws the line between what is feasible and what is impossible.

• Algorithmic mathematics provides a language for talking about program behavior.

• The lessons of program performance generalize to other computing resources.

• Speed is fun!

Running TimeRunning Time

• The running time depends on the input: an already sorted sequence is easier to sort.

• Parameterize the running time by the size of the input, since short sequences are easier to sort than long ones.

• Generally, we seek upper bounds on the running time, because everybody likes a guarantee.

Kinds of analysesKinds of analyses

Worst-case: (usually)• T(n) = maximum time of algorithm on any input of size n.

Average-case: (sometimes)• T(n) = expected time of algorithm over all inputs of size n.• Need assumption of statistical distribution of inputs.

Best-case:• Cheat with a slow algorithm that works fast on some input.

Machine-Machine-IIndependent timendependent time

The RAM Model

Machine independent algorithm design depends on a hypothetical computer called Random Acces Machine (RAM). Assumptions:• Each simple operation such as +, -, if ...etc takes exactly one time step.• Loops and subroutines are not considered simple operations.• Each memory acces takes exactly one time step.

Machine-independent timeMachine-independent time

What is insertion sort’s worst-case time?

• It depends on the speed of our computer,• relative speed (on the same machine),• absolute speed (on different machines).

BIG IDEA:• Ignore machine-dependent constants.• Look at growth of “Asymptotic Analysis”

nnT as )(

Machine-independent time: An exampleMachine-independent time: An example

A pseudocode for insertion sort ( INSERTION SORT ). INSERTION-SORT(A)

1 for j 2 to length [A]2 do key A[ j] 3 Insert A[j] into the sortted sequence A[1,..., j-1].4 i j – 15 while i > 0 and A[i] > key6 do A[i+1] A[i]7 i i – 18 A[i +1] key

Analysis of INSERTION-SORT(contd.)Analysis of INSERTION-SORT(contd.)

)1(][]1[6

10]11[ sequence

sorted theinto][Insert 3

timescost SORT(A)-INSERTION

nckeyiA

tciAiA

tckeyiAandi

ncjAkey

ncAlengthj

)1()1()1()(2

jj tctcncnccnT

).1()1( 82

The total running time is

The best case: The array is already sorted. (tj =1 for j=2,3, ...,n)

)1()1()1()1()( 85421 ncncncncncnT

).()( 854285421 ccccnccccc

•The worst case: The array is reverse sorted

(tj =j for j=2,3, ...,n).

)12/)1(()1()( 521 nncncncnT

)1()2/)1(()2/)1(( 876 ncnncnnc

ncccccccnccc )2/2/2/()2/2/2/( 87654212

cbnannT 2)(

Growth of FunctionsGrowth of Functions

Although we can sometimes determine the exact running time of an algorithm, the extra precision is not usually worth the effort of computing it.

For large inputs, the multiplicative constants and lower order terms of an exact running time are dominated by the effects of the input size itself.

Asymptotic NotationAsymptotic Notation

The notation we use to describe the asymptotic running time of an algorithm are defined in terms of functions whose domains are the set of natural numbers

...,2,1,0N

O-notationO-notation

• For a given function , we denote by the set of functions

• We use O-notation to give an asymptotic upper bound of a function, to within a constant factor.

• means that there existes some constant c s.t. is always for large enough n.

)(ng ))(( ngO

allfor )()(0

s.t.and constants positiveexist there:)())((

nnncgnf

ncnfngO

))(()( ngOnf

)(ncg)(nf

ΩΩ--Omega Omega notationnotation

• We use Ω-notation to give an asymptotic lower bound on a function, to within a constant factor.

• means that there exists some constant c s.t.

is always for large enough n.

)(ng ))(( ng

allfor )()(0

s.t.and constants positiveexist there:)())((

nnnfncg

ncnfng

))(()( ngnf

)(nf )(ncg

--Theta Theta notationnotation

• A function belongs to the set if there exist positive constants and such that it can be “sand- wiched” between and or sufficienly large n.

• means that there exists some constant c1 and c2 s.t. for large enough n.

)(ng ))(( ng

allfor )()()(c0

s.t.and,, constants positiveexist there:)())((

nnngcnfng

nccnfng

)(nf ))(( ng1c 2c

)(1 ngc )(2 ngc

))(()( ngnf )()()( 21 ngcnfngc

Asymptotic notationAsymptotic notation

Graphic examples of and . ,, O

1ncnnnc

Example 1. Example 1.

Show that

We must find c1 and c2 such that

Dividing bothsides by n2 yields

1)( 22 nnnnf

1,7 22

0 nnnn

Theorem Theorem

• For any two functions and , we have

if and only if

))(()( ngnf

)).(()( and ))(()( ngnfngOnf

Because :

)2(5223 nnn

Example 2.Example 2.

)2(5223)( nnnnf

)2(5223 nOnn

610033,3forsince)(61003 2222 nnncnOnn

3when61003,1forsince)(61003

610033,3forsince)(610032332

nnnncnOnn

nnncnOnn

cnncncnOnn

nnnncnOnn

nnncnOnn

when3,any forsince)(61003

610033,3forsince)(61003

nnnncnnn

cnncncnOnn

nnnncnOnn

nnncnOnn

610033,3forsince)(61003

nnnncnnn

cnncncnOnn

nnnncnOnn

nnncnOnn

100when61003,any forsince)(61003

610033,3forsince)(61003

nnncncnnn

nnnncnnn

cnncncnOnn

nnnncnOnn

nnncnOnn

apply. and both since)(61003

610033,3forsince)(61003

nnncncnnn

nnnncnnn

cnncncnOnn

nnnncnOnn

nnncnOnn

applies. only since)(61003

610033,3forsince)(61003

nnncncnnn

nnnncnnn

cnncncnOnn

nnnncnOnn

nnncnOnn

applies. only since)(61003

610033,3forsince)(61003

nnncncnnn

nnnncnnn

cnncncnOnn

nnnncnOnn

nnncnOnn

oo-notation-notation

• We use (small o) o-notation to denote an upper bound that is not asymptotically tight.

• We formally define as the set))(( ngo

allfor )()(0

s.t.0 constants aexist there

0constant positiveany for :)(

nnncgnf

isThat

ally tightasymptoticnot Is:)(2)(

ally tightAsymptotic:)(2)(

ωω-notation-notation

• We use ω-notation to denote an upper bound that is not asymptotically tight.

• We formally define as the set))(( ng

allfor )()(0

s.t.0 constants aexist there

0constant positiveany for :)(

nnnfncg

=relation The

∞→ g(n)

f(n)lim

tmplies thaω(g(n)) i f(n)

Example Example

Standard notations and common functionsStandard notations and common functions

• Floors and ceilings

11 xxxxx

• Modular arithmetic

For any integer a and positive integer n

nnaana /mod

• Polynomials:

Given a nonnegative integer d, a polynomial in n of degree d is

i nanp0

• Exponentials:

132 xx

• Logarithms:

)lg(lglglg

)(loglog

loglg 2

• Logarithms:

For all real a>0, b>0, c>0, and n

loglog

loglog)(log

• Logarithms:

log)/1(logloglog

• Series expansion:

)1ln(5432 xxxx

• Factorials

For the Stirling approximation:

)lg()!lg(

Designing algorithms Designing algorithms

There are many ways to design algorithms:

• Insertion sort uses an incremental approach• Merge sort uses divide-and-conquer approach

Insertion sort analysisInsertion sort analysis

Merge SortMerge Sort

MERGE_SORT(A,p,r)

1if p<r2 then q← (p+r)/2 3 MERGE_SORT(A,p,q)4 MERGE_SORT(A,q+1, r)5 MERGE(A,p,q,r)

Merging two sorted arraysMerging two sorted arrays

Analyzing merge sortAnalyzing merge sort

Recurrence for merge sortRecurrence for merge sort

Recursion treeRecursion tree

Introduction to Algorithms Lecture 1. Introduction The methods of algorithm design form one of the...

Documents

Transcript of Introduction to Algorithms Lecture 1. Introduction The methods of algorithm design form one of the...

UNIT - 3: Fast-Fourier-Transform (FFT) algorithms[ · Fast Fourier Transform Algorithms Introduction Computation of DFT DIT DFT Algorithm Radix-2 FFT Algorithm DFT as Linear Filtering

(Algorithms in Bipartite Graphs). Introduction Algorithms in unweighted bipartite graph (Yehong & Gordon) Maximum matching A simple algorithm Hopcroft-Karp.

Algorithm Design COMPSCI 311 Introduction to Algorithms ...

Skiena algorithm 2007 lecture01 introduction to algorithms

ADVANCED ALGORITHM ANALYSIS LECTURE # 1 INTRODUCTION-H euristic Algorithms

Algorithms and Data Structures. . . . . .3/27 Introduction Algorithms Data Structures Insertion Sort Analysis of the Algorithm Algorithms Informally, an algorithm is any well-deﬁned

Introduction to Contextual Multi-bandit Algorithm to Contextual Multi-bandit... · Outline §Introduction §Motivation §Contextual-free Bandit Algorithms §Contextual Bandit Algorithms

An Algorithm for: Explaining Algorithms

MULTIGRID ALGORITHMS - Freeebrary.free.fr/Mesh Generation/Wesseling - Introduction to... · 8 MULTIGRID ALGORITHMS 8.1. Introduction ... The linear multigrid algorithm follows easily

Lecture # 1 Introduction Analysis of Algorithm by Qamar Abbas Analysis of Algorithms.

CSE 780 Algorithms Advanced Algorithms Minimum spanning tree Generic algorithm Kruskal’s algorithm Prim’s algorithm.

COMPSCI 311: Introduction to Algorithms What is Algorithm ...

Introduction to Algorithms and Programming...2 1 Introduction to Algorithms What is an algorithm? Our text defines an algorithm to be any well-defined computational procedure that

Algorithm Animation for Bioinformatics Algorithms.

INTRODUCTION TO ALGORITHMS PROGRAMMING. Objectives Give a definition of the term algorithm Describe the various parts of the pseudocode algorithm or algorithm.

Clustering Genetic Algorithm - avcr.cz · ETID’2007 Introduction Clustering Genetic Algorithm Experimental results Conclusion Genetic algorithms Genetic algorithms stochastic optimization

An introduction to randomized algorithms - Nicole · PDF fileAn introduction to randomized algorithms ... randomized primality test developed by Solovay ... algorithm that performs

CS6212-10, Computing Algorithms I Lecture Notehchoi/teaching/cs6212d/noteall.pdfCS6212-10, Computing Algorithms I Lecture Note 1 Introduction •Algorithm – What is algorithm? –

Genetic Algorithms: An Introduction - curaj.ac.in Algorithm Introduction.pdf · What are Genetic Algorithms Inspired by Biological Evolution Search and Optimization Procedures Good

COT 6405 Introduction to Theory of Algorithmsyliu21/Algorithm/Lecture 17.pdf · COT 6405 Introduction to Theory of Algorithms ... –To develop approximation algorithms, instead of