ANALYSIS OF ALGORITHMS

Data Structures

Algorithms

DefinitionA step by step procedure for performing some task in a finite amount of time

Analysis Methodology for analyzing a “good” algorithm (i.e. Archimedes)

Look at Running Time

How are algorithms described ? PSEUDO-CODE

representation of an algorithm combines

natural languagefamiliar programming language

structuresfacilitates the high-level analysis of

an algorithm

Pseudo-Code Conventions

ExpressionsAlgorithm StructuresControl Structures

Expressions

Standard math symbols + - * / ( )

Relational operatorsBoolean operators

and or not

Assignment operator , =Array indexing A[i]

Algorithm Structure

Algorithm headingAlgorithm name(param1, param2,...):Input : input elementsOutput : output elements

Statementscall object.method(arguments)return statement return valuecontrol structures

Control Structures

decision structures

while loops

repeat loops

for loop

if ... then ... [else ...]if ... then ... [else ...]

while ... dowhile ... do

repeat ... until ...repeat ... until ...

for ... dofor ... do

General Rules

communicate high level ideas and not implementation details (programming language specifics)

clear and informative

Example - Problem

Problem : Write pseudo-code for an algorithm that will determine the maximum element from a list (array)

Example - Algorithm

Algorithm arrayMax(A,n):Input: An array A storing n integers.Output: The maximum element in A. currentMax A[0] for i 1 to n - 1 do if currentMax < A[i] then currentMax A[i] return currentMax

Analysis of Algorithms

Running timedepends on input size nnumber of primitive operations

performedPrimitive operation

unit of operation that can be identified in the pseudo-code

Primitive Operations-Types

Assigning a value to a variableCalling a method/procedurePerforming an arithmetic operationComparing two numbersReturning from a method/procedure

Primitive Operations - Some General Rules

for LoopsNested for LoopsConditional Statements

for Loops

The running time of a for loop is at most the running time of the statements inside the for loop (including tests) times the number of iterations.

Example for i 0 to n - 1 do A[i] 0

Nested for Loops

Analyze these inside out. The total running time of a statement inside a group of nested for loops is the running time of the statement multiplied by the product of the sizes of all the for loops. Example for i 0 to n - 1 do for j 0 to n - 1 do

k++

Conditional Statements

if (cond) then S1

else S2

The running time of an if-else statement is never more than the running time of the test plus the larger of the running times of S1 and S2.

Primitive Operations - Example

Count the number of primitive operations in this program : i = 0 a = 0 for i = 1 to n do

print i a=a+i

return i

Primitive Operations - Example

Estimate : 3n + 3Linear Time

- (i.e. 6n + 2, 3n+5, n+1,etc.)

Primitive Operations - Example 2

Count the number of primitive operations in this program :

i = 0a = 0

for i = 1 to n doprint i for a = 1 to n do

print areturn i

Primitive Operations - Example2

Estimate : n2 + n + 3Polynomial (Quadratic) Time

Example 3 - Class

Algorithm sum(int n):Input: Integer nOutput: Sum of the cubes of 1 to n partial_sum 0 for i 1 to n do

partial_sum partial_sum + i*i*i return (partial_sum)

Primitive Operations -Considerations

Arbitrariness of measurementWorst-case versus average-case

(difficult to be exact : range)Implicit operations

loop control actionsarray access

Analysis of Algorithms

take-out the effect of constant factors (i.e. 2n vs 3n : same level)

this can be attributed to differences in hardware and software environments

compare algorithms across types (linear vs polynomial vs exponential)

look at growth rates

Asymptotic Notation

Goal:To simplify analysis by getting rid of

irrelevant information

The “Big-Oh” NotationGiven functions f(n) and g(n), f(n) is O

(g(n)) if f(n) <= c g(n) for n>= n0 where c and n0 are constants

Big-Oh NotationRule : Drop lower order terms and

constant factors5n + 2 is O (n)4n3log n + 6n3 + 1 is O (n3logn)

Big-Oh Notation - Relatives

Big Omega (reverse)Big Theta (both ways, same level)

General Rule : Use the most descriptive notation to describe the algorithm

Practical Significance of Big-Oh Notation

Example for i 0 to n-1 do A[i] 0 Running time is

2n, if we count assignments & array accesses3n, if we include implicit loop assignments4n, if we include implicit loop comparisons

Regardless, running time is O (n)

Algorithms - Common Classifications

Typical functions that classify algorithmsconstant O (1)logarithmic O (log n)linear O (n)quadratic O (n2)exponential O (an), n > 1

Linear Search vs Binary Search

Linear Search O (n)Binary Search O (log n)

Example : to search for a number from 1024 entries using the binary search algorithm will only take 10 steps

O(log n) Algorithm

Steps Range (n) Power of 20 1 20

1 2 21

2 4 22

3 8 23

4 16 24

5 32 25

6 64 26

7 128 27

O(log n) Algorithm Binary Search

Algorithm bin_search(int a[], int x, int n)

Input: Array a, search target x, array size n

Output: Position of x in the array (if found)

`{ low 0; high n - 1while (low <= high) do mid (low + high)/2

if(a[mid] < x) then low mid + 1 else if (a[mid] > x) then high mid - 1 else return (mid)return (NOT_FOUND)}

Prefix Average Problem

Given an array X storing n numbers, we want to compute an array A such that A[1] is the average of elements X[0] … X[I] for I = 0 … n-1.

algorithm A = O (n2) algorithm B = O (n)