Complexity analysis

Complexity Theory

Complexity theory is a problem can be solved? Given:◦ Limited resources: processor time and memory

space

Complexity Theory

Algorithm

Definition: An algorithm is a finite step-by-step list of well-defined instructions for solving a particular problem.

Algorithm complexity: is a key task in comparing the efficiency of algorithms.

Two factors are considered:◦ 1- the (running) time Time Complexity◦ 2- the space (usage) Space Complexity

Algorithm

Time Complexity: estimates the running time of the algorithm.◦ Measured by the number of key step of the

basic steps used by the algorithm. Basic Steps are:

◦ Arithmetic operations (+, -, *, /)◦ Comparison operations (>, >=, <, <=, ==)◦ Reading from a memory, file◦ Writing to memory, file, (cin, fin)

Time Complexity

Key step: The most costly step of the algorithm◦ Takes most of the running time◦ Other steps is much less or at most proportional

to the time of the key step.

Key step

In worse-case analysis:◦We find the maximum value of the

number of the key steps.

In Average-case analysis:◦We find the average value of the number

of the key steps.

Worse-Case & Average-Case Analysis

Let f (n) the complexity of an algorithm, i.e.,◦ # key operations◦ For an instance of size n

A computer processor has a speed of 1 MIP◦ Million Instructions Per (MIP) second

Growth Rate

Growth Rate (Cont.)

Big Oh O() Notation is used◦ To estimate the growth rate of f : N R+

◦ f ϵ O(g)◦ There exist two positive integer k, n0 :

f(n) ≤ k*g(n), for all n ≥ n0

f grows not fast than g f belongs to the complexity class O(g)

Big Oh O( ) Notation

f(n) = 3 n+2= O(n) as 3 n +2 ≤ 4 n for all n ≥ 2.

f(n) =100 n+6=O(n) as 100 n+6 ≤ 101 n for all n ≥ 6.

f(n) =1000 n2 + 100 n - 6 = Ο(n2) as 1000 n2 + 100 n - 6 ≤ 1001 n2 for all n ≥ 100.

f(n) = 6 2n+ n2 = Ο (2n) as 6 2n+ n2 ≤ 7 * 2n for all n ≥ 4.

Big Oh O ( ) Examples

Find the class complexity of the following functions

1. f(n) = n3 + 23n2 - 8n + 6 2. f(n) = (6n7+4) / (n2+1) 3. f(n) = 2n + 3n5 + 34n4

Let’s practice!

Common Complexity Classes

Common complexity classes

O(1)< O(log n) < O(n)< O(n log n)< O(n2)<O(n3)< O(2n)< O(10n)

O(n log n) is better than O(n2) O(n) is better than O(n log n)

Class Complexity Hierarchy

Dropping Coefficient◦f(n) = k * nc f ϵ O(nc)◦f(n) = 10 n4 f ϵ O(n4)

Dominant Term◦f(n) = nc+nc-1+…+ n2 +n+1 f ϵ O(nc)◦f(n) = (n8 + 1) / (n3 + 2n2 – 1) f ϵ O(n5)

Manipulating Big Oh O() Notation

Introduction to Algorithms

What is Algorithm ??

An algorithm: is a finite step-by-step list of well-defined instructions for solving a particular problem.

Algorithm complexity is a key task in comparing the efficiency of algorithms.

Two factors are considered:◦ 1- the (running) time Time Complexity◦ 2- the space (usage) Space Complexity

Algorithm

Real life algorithm

Sorting Algorithms◦ Bubble Sort◦ Insertion Sort◦ And many other algorithms (Quick Sort, Selection Sort, ….

)

Searching Algorithms◦ Linear search (sequential search)◦ Binary search◦ And many other algorithms (BFS, DFS, … )

And There are many and many algorithms used in this fields and other fields.

Examples of algorithms

Sorting Algorithms

Bubble sort

Insertion Sort