DESIGN AND ANALYSIS OF PREREQUISITE ALGORITHMS...17CS304 Data Structures II-I 17CS403 Advanced Data...

DAA UNIT-I Chapter-1

Blog @ anilkumarprathipati.wordpress.com 1

DESIGN AND ANALYSIS OF ALGORITHMS

Prepared by

Anil Kumar Prathipati, Assistant Professor, Website/blog: anilkumarprathipati.wordpress.com

Raghu Engineering College(Autonomous)

UNIT – I Chapter -1

COURSE CODE

COURSE NAME YEAR -SEM

17CS303 Mathematical Foundations of Computer Science

II-I

17CS304 Data Structures II-I

17CS403 Advanced Data Structures II-II

PREREQUISITE

Problem

A1 A4 . . . .A3A2 AmA3

P4 . . . .P2P1 P3 Pn

Algorithms

Programs

After Analyzing

Algorithm + Data Structure = Program

Solutions

Art of the Computing

Why Study this Course?

Examples – Algorithms are everywhere Dictionaries

Google Maps

E-commerce websites

Search engines

Video streaming

Softwares

Contd ...



Contd ...

Example: Consider the Sorting Problem

Two Algorithms: 1) Navie based Algorithm (like brute force) ==> O(n

2)

2) Best Algorithm ==> O(n log n)

Coming to System Performance (Speed)CPU with speed (1 Ghz) 10

9operations per second

Problem is to sort around 109

Phone numbers

By using Algorithm -1, What is the total time it takes?

Algorithm -2, What is the total time it takes?

Q): Which Algorthm do you choose?

Contd ...

Example: Consider the Computer GAMELet, If there are Five Lakhs (5X10

5) objects and all

objects are moving(screen resolution 1920 * 1080 more than 20Lakhs pixels) . If You are pointing at one object. If your game isbased on the nearest path from your pointing object. Consider theprevious CPU speed.

Two Algorithms: 1) Navie based Algorithm (like brute force) ==> O(n

2)

2) Best Algorithm ==> O(n log n)

By using Algorithm -1, How much time does it takes?

Algorithm -2, How much time does it takes?

Q): Which Algorthm do you choose?

UNIT DETAILS

I Introduction: Examples and motivation, Asymptotic complexity: informalconcepts, formal notation, examplesSearching and Sorting: binary search, insertion sort, selection sort, mergesort, quick sort, stability and other issues.

II Graphs: Motivation, Directed acyclic graphs, Graph exploration: BFS, DFS,applications.

III Search Trees: Introduction, Traversals, insertions, deletions, Balancing,Priority queues, heaps

IV Greedy: Interval scheduling, Minimum cost spanning trees: Prim’s algorithm,Kruskal’s Algorithm, Shortest paths: un-weighted and weighted, Singlesource shortest paths: Dijkstra’s, Huffman coding.

SYLLABUS Text Book



UNIT-I SYLLABUS

Introduction: Examples and motivation, Asymptoticcomplexity: informal concepts, formal notation,examples.Searching and Sorting: binary search, insertion sort,selection sort, merge sort, quick sort, stability and otherissues.

Reference:“Fundamentals of Computer Algorithms” by Horowitz, Sahniand Rajasekaran.

What is an Algorithm?

• Name of a Persian mathematician Abu Jafar MohammedIbn Musa Al Khowarizmi (ninth century).

• Finite set of instructions that accomplishes a particulartask.

or• sequence of unambiguous instructions for solving a

problem

1. Characteristics / Properties

Finiteness Definiteness Correctness Output Input Effectiveness

An algorithm is a finite sequence of unambiguous instructions(steps) for solving a problem. i.e. for obtaining a required outputfor any legitimate (valid) input in a finite amount of time.

Motivation Example Air travel



2. Process for DAA

Fig: Algorithm Design and Analysis Process

Understand the Problem

Decide on: Computational means, Algo. Design technique

Design an Algorithm

Prove Correctness

Analyze the Algorithm

Code the algorithm

Yes

Yes

No

No

2. Process for DAA1)Understand the problem: It is crucial step. So before designing an algorithm is to

understand the problem first.2)Decide on: Study the architecture of the machine and supporting software tools. Solution

as an algorithm (Exact vs approximation solving). Solve the problem exactly if possible.3)Algorithm design techniques: we will use different design techniques like,

Divide-and-conquer Greedy method Dynamic programming Backtracking Branch and bound ....etc...

4) Prove correctness: Testing is used for proving correctness.5) Analyze an algorithm: Calculating the time complexity and space complexity.6) Coding an algorithm: after completion of all phases successfully then we will

code an algorithm.

Ultimately algorithms are implemented as computer programs.

Contd ...

There are various issues in the study of algorithms.

1)How to devise an algorithm: may never be fully automated.2)How to express algorithm: using the best principles of structuring.3)How to validate algorithm: The purpose of validation of algorithm

is to find whether algorithm works properly without being dependentupon programming languages.

4)How to analyze algorithm: The behavior of algorithm in best case,worst case and average case needs to be obtained.

5)How to test a program: Testing a program really consists of twophases:

Issues for algorithm 3. Algorithm Types

1) Approximate algorithm.2) Probabilistic algorithm.3) Infinite algorithm.4) Heuristic algorithm.



1)Approximate Algorithm: An algorithm is said to approximate if it is

infinite and repeating. Ex: squareroot(2), etc..

2)Probabilistic algorithm: If the solution of a problem is uncertain then it

is called as probabilistic algorithm. Ex: Tossing of a coin.

3)Infinite Algorithm: An algorithm which is not finite is called as infinite

algorithm. Ex: A complete solution of a chessboard, division by zero.

4)Heuristic algorithm: Giving fewer inputs getting more outputs is called

the Heuristic algorithms. Ex: All Business Applications.

Contd ...

4. Algorithm Specification

AlgorithmPseudocode

Flowchart

Program

Natural Language

Using Natural Language: It is very easy to specify an algorithm using natural

language. But many times specification of algorithm by using natural

language is not clear, and may require brief description.

Eg: Write an algorithm to perform addition of two numbers.Step 1: Read the first number, say ‘a’.Step 2: Read the second number, say ‘b’.Step 3: Add the two numbers and store the result in a

variable ‘c’.Step 4: Display the result.

Contd ...

Flowchart Flow chart is a graphical representation of an algorithm. But flowchart method work well only if the algorithm is small and

simple.

Contd ...



Pseudo-Code for Expressing Algorithms

1.Comments begin with // and continue until the end of line.

1.A block of statements (or) compound statements are representedusing { and } for example if statement, while loop, functions etc.,.

3.The delimiters [;] are used at the end of the each statement.

Example{Statement 1;Statement 2;..................}

4. An identifier begins with a letter. Example: sum, sum5, a; but not in5sum, 4a etc.,.

4. Assignment of values to the variables is done using the assignmentoperators as := or

6. There are two Boolean values TRUE and FALSE.

Logical operators: AND, OR, NOTRelational operators: <, , ≥,,=,Arithmetic operators: +, -, *, /, %

Contd ...

7. The conditional statement if-then or if-then-else is written in thefollowing form.

If (condition) then (statement)If (condition) then (statement-1) else (statement-2)If a condition is true the particular block of

statements are execute.

Contd ...

Exampleif(a>b) then{write("a is big");}else{write("b is big");}

8. The Case statement

8. Loops for

Contd ...

case{:(condition -1): (statement-1):(condition -2): (statement-2):(condition -n): (statement-n)............................else :(statement n+1);}

for variable:=value 1 to value n step do{Statement -1;Statement -1;..............Statement -n;}

Example:for i:=1 to 10 do{write(i); //displaying numbers from 1 to 10i:=i+1;}



While

Repeat -until

Contd ...

while <condition> do{<statement 1><statement 2>................<statement n>}

Example:i:=1;while(i<=10)do{write (i);//displaying numbers from 1 to 10i:=1+1;}

repeat{<statement 1><statement 2>............<statement n>}until <condition>

Examplei:=1;repeat{write (i);i:=i+1;}until (i>10);

10. Break: this statement is exit from the loop.

10. Elements of array are accessed using [ ].Ex: if A is an one-dimensional array, then A[i]. If A is two-

dimensional array, A[i,j].

12. Procedures (functions):

Algorithm Name (<parameter list>){ body of the procedure}

procedure name of the procedure

Syntax:

Contd ...

13. Compound data-types can be formed with records

Contd ...

Name = record{data-type -1 data 1;data-type -2 data 2;data-type -n data n;}

ExampleEmployee =record{int no;char name[10];float salary;}

Algorithm Pseudocode

An unambiguousspecification of how to solvaa problem

An informal high-leveldescription of the operatingprinciple of a computerprogram or other algorithm

Helps to simplify andunderstand the problem

A method of developing analgorithm

Algorithm vs Pseudocode



5. Performance Analysis

• To determine resource consumption– CPU time– Memory space

• Compare different methods for solving the same problembefore actually implementing them and running theprograms.

• To find an efficient algorithm

Q: Need for analysis?

Complexity

A measure of the performance of an algorithm

Algorithm’s performance depends on internal factors external factors

• Speed of the computer on which it is run• Quality of the compiler• Size of the input to the algorithm

Internal Factor

• The algorithm’s efficiency, in terms of: Time required to run Space (memory storage)required to run

Note:Complexity measures the internal factors (usually moreinterested in time than space)

External Factor Space Complexity

• The space needed by an algorithm is the sum of a fixed part and avariable part.– Space complexity S(P) = C + Sp (Instance characteristics)

• The fixed part includes space for– Instructions– Simple variables– Fixed size component variables, etc..

• The variable part includes space for– Component variables whose size is dependant on the particular problem

instance being solved– Recursion stack space, etc..



Examples

1) Algorithm sum (int x, int y){

int sum;sum = x+y;

Return (sum);}

2) Algorithm sum (int x, int y){

Return (x+y);}

3) Algorithm fun (float x, float y, float z){

Return (X + Y +Y * Z + (X + Y +Z)) /(X+ Y) + 4.0;

}

4) Algorithm sumarr (int a[], int n){

int *b,i;b=(int*)malloc(n*sizeof(int));for(i =0;i<n;i++)

{b[i] = a[i];

}}

Examples5) Algorithm ADD ( float [], int n)

{sum := 0.0;for i:=1 to n do

{sum:=sum + X[i];

}return sum; }

6) Algorithm COPY2D ( float a[][], int n){int i,j,b[][10];for i:=1 to r do

{for j:=1 to c do

{b[i][j] = a[i][j]}

}return b;

}

7)

1. int f (int n) 2. {3. if (n >= 1) 4. {5. f(n-1);6.

Display (n);7. f(n-1);8. }9. }

10. int main()11. {12. f(2);13. }

f (2) {if (2 >= 1) //TRUE

{Display (2);

f(2-1);}

}

f (1){

if (1 >= 1) //TRUE{

Display (1);f(1-1);

}}

f (0){

if (0 >= 1) //FALSE{

// not consider}

}

n = 2Return

Address(Line #11)

n = 1Return

Address(Line #5)

n = 0Return

Address(Line #5)

Output: 1 2

Parameters Local Variables Return Address

Fig: Activation Record

Time Complexity

• The time complexity of a problem is– The number of steps that it takes to solve an instance of the problem

as a function of the size of the input (usually measured in bits), usingthe most efficient algorithm.

– Priori analysis or compile time

– Posteriori analysis or run (execution) time.

• Time complexity T(P) = C + T (n)– It can be two measured in two ways:

1. By using Tabular Method2. By using Global count variable



Two ways of finding complexity

• Experimental study• Theoretical Analysis

Experimental Study

• Write a program implementing the algorithm

• Run the program with inputs of varying size and composition

• Get an accurate measure of the actual running time

• Plot the results

Limitations of Experiments

• It is necessary to implement the algorithm, which may be difficult.

• Results may not be indicative of the running time on other inputsnot included in the experiment.

• In order to compare two algorithms, the same hardware andsoftware environments must be used.

• Experimental data though important is not sufficient.

Theoretical Analysis

• Uses a high-level description of the algorithm instead ofan implementation

• Characterizes running time as a function of the input size,n.

• Takes into account - all possible inputs• Allows us to evaluate the speed of an algorithm

independent of the hardware/software environment



1. Global variable count method:(Operation Count)

Contd ...

Algorithm Sum(a, n){

s:=0;for i:=1 to n do{

s:=s+a[i];}

return s;}

count:=0;Algorithm Sum(a,n){

s:=0;count:=count+1;for i:=1 to n do{

count:=count +1;s:=s+a[i];count:=count+1;

}count:=count+1; //for last time of for

loop

return s;count:=count+1; //for return statement

}

Thus the total number of steps are 2n+3

2. Tabular method (Step Count):

Contd ...

Ex:

Time complexity cases

• Best Case: Inputs are provided in such a way that the minimum time isrequired to process them.

• Average Case: The amount of time the algorithm takes on an average setof inputs.

• Worst Case: The amount of time the algorithm takes on the worstpossible set of inputs.

3 4 5 6 7 9 10 12 15

Linear Search

A 1 2 3 4 5 6 7 8 9

Example:

6. Asymptotic Notation

• The exact number of steps will depend on exactly whatmachine or language is being used.

• To avoid that problem, the Asymptotic notation isgenerally used.

• Running time of an algorithm as a function of input sizen for large data n.

• Expressed using only the highest-order term in theexpression for the exact running time.



Big - Oh notation: (O)

Let f(n) and g(n) be the two non-negativefunctions. We say that f(n) is said to beO(g(n)) if and only if there exists apositive constant ‘c’ and ‘n0‘ such that,f(n)c*g(n) for all non-negative values ofn, where n≥n0.

Here, g(n) is the upper bound for f(n).

Big - Omega notation:

Let f(n) and g(n) be the two non-negativefunctions. We say that f(n) is said to be(g(n)) if and only if there exists apositive constant ‘c’ and ‘n0‘ such that,f(n) ≥ c*g(n) for all non-negative valuesof n, where n≥n0.

Here, g(n) is the lower bound for f(n).

Big - Theta notation:

Let f(n) and g(n) be the two non-negetive functions. We say that f(n) issaid to be (g(n)) if and only if thereexists a positive constants ‘c1’ and ‘c2’,such that, c1g(n) f(n) c2g((n) for allnon-negative values n, where n ≥ n0.

The above definition states that the function f(n) lies between ‘c1’times thefunction g(n) and ‘c2’, times the function g(n) where ‘c1’ and ‘c2’ are positiveconstants.

Big-Oh vs Big-Omega• Big oh notation is denoted by ‘O’, whereas Omega notation is denoted by

‘’.• Big oh is used to represent the upper bound of an algorithm’s running time,

i.e. we can give largest amount of time taken by the algorithm to complete.But whereas omega notation represent the lower bound of an algorithm’srunning time, i.e. we can give smallest amount of time taken by thealgorithm to complete.

• Let f(n) and g(n) be the two non-negative functions. We say that f(n) is saidto be O(g(n)) if and only if there exists a positive constant ‘c’ and ‘n0‘ suchthat, f(n)c*g(n) for all non-negative values of n, where n≥n0. But whereasfunction f(n)= (g(n)) (read as for of n is omega of g of n) if and only if thereexist positive constants ‘c’ and ‘n0’ such that, f(n) ≥ c*g(n) for all n, n≥n0.



Asymptotic Notation Properties

• General property

• Transitive

• Reflexive

• Symmetry

• Transpose Symmetry

General property

• If f(n) is O(g(n)) then a*f(n) is O(g(n))• If f(n) is Θ(g(n)) then a*f(n) is Θ(g(n))• If f(n) is Ω(g(n)) then a*f(n) is Ω(g(n))for any ‘a’

Eg: f(n) = 2n2+5 is O(n2). Let as assume a= 7

Transitive “Given three functions, f,g and h, we expect that ordering is

transitive: if h ≤ g and g ≤ f then h ≤ f”

• If f(n) = Θ(g(n)) and g(n) = Θ(h(n)), then f(n) = Θ(h(n))• If f(n) = O(g(n)) and g(n) = O(h(n)), then f(n) = O(h(n))• If f(n) = Ω(g(n)) and g(n) = Ω(h(n)), then f(n) = Ω(h(n))

Eg: f(n) = n, g(n) = n2 , h(n)= n3

Reflexivity

“The reflexive property states that any real number, a, is equal toitself. That is, a = a.”

• f(n) = Θ(f(n))• f(n) = O(f(n))• f(n) = Ω(f(n))

Eg: f(n) = n2 is O(n2)



Symmetry

“This property states that if a = b, then b=a”

• f(n) = Θ(g(n)) if and only if g(n) = Θ(f(n))

• Eg: f(n) = n2 , g(n) = n2

Transpose Symmetry

• f(n) = O(g(n)) if and only if g(n) = Ω(f(n))

Eg: f(n) = n , g(n) = n2

If f(n) = O(g(n)) and d(n)=O(e(n))then f(n) + d(n) = O( max( g(n), e(n) ))Example: f(n) = n i.e O(n)d(n) = n² i.e O(n²)then f(n) + d(n) = n + n² i.e O(n²)

If f(n)=O(g(n)) and d(n)=O(e(n))then f(n) * d(n) = O( g(n) * e(n) )Example: f(n) = n i.e O(n)d(n) = n² i.e O(n²)then f(n) * d(n) = n * n² = n³ i.e O(n³)

Some more properties

Time complexity Example

O(1) constant Adding to the front of a linked list

O(log N) log Finding an entry in a sorted array

O(N) linear Finding an entry in an unsorted array

O(N log N) n-log-n Sorting n items by ‘divide-and-conquer’

O(N2) quadratic Shortest path between two nodes in a graph

O(N3) cubic Simultaneous linear equations

Common growth rates



Q) What is the time complexity of fun()?int fun(int n){int count = 0;for (int i = 0; i < n; i++)

for (int j = i; j > 0; j--)count = count + 1;

return count;}(A) Theta (n)(B) Theta (n^2)(C) Theta (n*Logn)(D) Theta (nLognLogn)

Q) What is the time complexity of fun()? [2012]

int fun(int n) { int count = 0; for (int i = n; i > 0; i /= 2)

for (int j = 0; j < i; j++) count += 1;

return count; }(A) O(n^2)(B) O(nLogn)(C) O(n)(D) O(nLognLogn)

Q) Consider the following three claims [2013]

1. (n + k)m = Θ(nm), where k and m are constants2. 2n + 1 = O(2n)3. 22n + 1 = O(2n)Which of these claims are correct?

(A) 1 and 2(B) 1 and 3(C) 2 and 3(D) 1, 2, and 3



Q) Consider the following functions from positives integers to real numbers

10, √n, n, log2n, 100/n.The CORRECT arrangement of the above functions in increasing

order of asymptotic complexity is: [2018]

(A) log2n, 100/n, 10, √n, n(B) 100/n, 10, log2n, √n, n(C) 10, 100/n ,√n, log2n, n(D) 100/n, log2n, 10 ,√n, n

Example: Matrix Multiplication

void multiply(int A[][N], int B[][N], int C[][N]) {

Step 1: for (int i = 0; i < N; i++){

Step 2: for (int j = 0; j < N; j++) {

Step 3: C[i][j] = 0;Step 4: for (int k = 0; k < N; k++) \{

Step 5: C[i][j] += A[i][k]*B[k][j];}

}}

}

Q: Find the Time and Space Complexity?

DESIGN AND ANALYSIS OF PREREQUISITE ALGORITHMS...17CS304 Data Structures II-I 17CS403 Advanced Data...

Documents

Transcript of DESIGN AND ANALYSIS OF PREREQUISITE ALGORITHMS...17CS304 Data Structures II-I 17CS403 Advanced Data...