Allmant

Complexity, General

Find a function T (n), which behaves as the time it takes to execute the program forinput of ’size’ n.

Standard approach: count the number of primitive operations executed.

Standard approximations: we are not interested of the precise count, only of theincrease ratio as a function of the ’size’ n.

For this we use the concept of Big O.

LAD, complexity 1

Example

Example

Compute the sum of the numbers from 1 to n

int sum = 0; ( 1 op)

for ( int i = 1; i <= n; i++ ) ( 1 op, 1op, 2 op)

sum =+ i; ( 2 op )

Gives the time function:

T (n) = 1 + 1 + (n + 1) ∗ 1 + n ∗ (2 + 2)

= 3 + 5 ∗ n

LAD, complexity 2

Big O

Definition of Big O

Let g be a function whose domain and co-domain are the

non-negative integer values.

A function f is said to be of O(g) if for some pair of

non-negative constants C and K, it holds:

(∀n ≥ K) f(n) ≤ C · g(n)

LAD, complexity 3

Big O

Common Big O functions

O(1) : The constant complexity

O(2log n) : The logaritmic complexity

O(n) : The linear complexity

O(n2) : The quadratic complexity

O(n3) : The cubic complexity

O(2n) : The exponential complexity

and combinations of the functions above, e.g. O(n ∗ 2log n)

LAD, complexity 4

Big O

Example above

Take the summation above for which we have f(n) == T (n) = 3 + 5 ∗ n.

Let g be the function g(n) = n.

Choose C = 6 and K = 3 then we have that:

f(n) ≤ C · g(n), for all n > K.

i.e. f is of O(g), or f ∈ O(g)

LAD, complexity 5

Big O

Example double loop

Consider the loop-statement:

for ( int i = 0; i < n; i++ )

for (int j = 0; j < n; j++ )

// k primitive operations

We find the complexity function by setting the work of the k primitive operations toone and use double summation:

n∑

i=0

n∑

j=0

1 =n∑

i=0

n = n ∗ n ∈ O( n2 )

LAD, complexity 6

Big O

Example: double loop

Now change the inner loop:

for ( int i = 0; i < n; i++ )

for (int j = i; j < n; j++ )

// k primitive operations

and we get

n−1∑

i=0

n−1∑

j=i

1 =n−1∑

i=0

n − i =n∑

i=1

i =n ∗ (n + 1)

2∈ O( n2 )

LAD, complexity 7

Big O

Example squareRoot

public static int squareRoot( int n ) {

int i = 1, j = n+1;

while ( i + 1 != j ) {

int k = ( i + j ) / 2;

if ( k*k <= n )

i = k;

else

j = k;

}

return i;

} // squareRoot

LAD, complexity 8

Big O

Complexity of squareRoot

The complexity varies depending on the size of the argument (n)

We get the time function:

T (n) = 1 + 2 + 2 + noOfLoops ∗ (2 + 3 + 2 + 1)

or shorter T (n) = 5 + 8 ∗ noOfLoops

Since the interval (i .. j )is decreased to half its size each loop,the number of loops becomes ⌈ 2log n ⌉

why we draw the conclusion T (n) ∈ O( 2log n )

LAD, complexity 9

Big O

Program squareRoot

public static int noOfLoops;

public static int squareRoot( int n ) {

while ( i + 1 != j ) {

noOfLoops++;

...

}

...

} // squareRoot

public static void main(String[] args ) {

int[] a = { 1, 8, 32, 128, 1024, 2048, 10000 };

for( int n : a ) {

noOfLoops = 0;

System.out.println( n + "\t" + squareRoot(n) + "\t" + noOfLoops

}

LAD, complexity 10

Big O

Run squareRoot

sed:bjerner:[~/dvds/program]$ java SquareRoot

1 1 1

8 2 3

32 5 5

128 11 7

1024 32 10

2048 45 11

10000 100 14

LAD, complexity 11

Insertion sort

Algorithm

Given an array a to be sorted.

let i = smallest index + 1while i ≤ highest index.

if a[i] < a[i − 1]let temp = a[i], a[i] = a[i − 1] and j = i − 1while j > smallest index and a[j − 1] > temp

let a[j] = a[j − 1]decrease j by 1

let a[j] = temp

increase a[i] by 1

LAD, complexity 12

Insertion sort

Correctness Predicates

Loop invariant: The segment to the left of i in a is sorted.

Loop condition: i is less than or equal to the highest index for a.

Loop conclusion: The whole array is sorted.

LAD, complexity 13

Insertion sort

Java code

public static void insertionSort( int[] a ) {

for ( int i = 1; i < a.length; i++ )

if ( a[i] < a[i-1] ) {

int tmp = a[i];

a[i] = a[i-1];

int j = i-1;

while ( j > 0 && a[j-1] > temp ) {

a[j] = a[j-1];

j--;

}

a[j] = temp;

}

}\\ insertionSort

LAD, complexity 14

Insertion sort

Complexity

In analysing the comlexity of programs depending on probabilities, we analyse:

the best case and the worst case.

if they have different Big O complexity, we also analyse the:

the average case

LAD, complexity 15

Insertion sort

Complexity

The best case: Its obviously when the array is already sorted.Only the if-condition is computed. If size n = a.length, we get:

n−1∑

i=0

1 = n ∈ O(n ), i.e. linear

The worst case: When the array is sorted reversely. Gives:

n−1∑

i=0

i∑

j=1

1 =n−1∑

i=0

i =n ∗ (n − 1)

2∈ O( n2 ), i.e. quadratic

LAD, complexity 16

Insertion sort

Complexity

The average case: Assume that the inner loop only goes halfways, then we get:

n−1∑

i=0

i2∑

j=1

1 = ... =N ∗ (N − 1)

4∈ O(n2 ), i.e. still quadratic

LAD, complexity 17

Stacks

Methods on stacks

Historically, a stack is an object on which we can perform following methods:

• push(element) : void, puts an element onto the stack.

• pop() : void, takes the last pushed element away from the stack,i.e. in FILO order (or in LIFO order)

• top() : element, gives the last pushed element,but let it stay in the stack

• isEmpty : boolean: gives true if the stack is empty, otherwise false

Sometimes size() : integer occurs.

Cmp: with a plate holder in a restaurant.

LAD, complexity 18

Stacks

Exemple, pre-, in-, and post-fix

In order to place a binary operator with respect to its operandes, there are threepossibilities:

• preorder, i.e. first, + op1 op2

• inorder, i.e. between, op1 + op2

• postorder, i.e. last, op1 op2 +

LAD, complexity 19

Stacks

Exemple A, expressions of different forms

We want to take the product of a and b and add it to the product of c and d

preorder : + ∗ a b ∗ c d

inorder : a ∗ b + c ∗ d

postorder : a b ∗ c d ∗ +

Note: For inorder we have to use priorty rules in order to perform the operations inright order !

LAD, complexity 20

Stacks

Exemple B, expressions of different forms

Now, in stead we want to take the sum of a and b and multiply it with the sum of c

and d

preorder : ∗ + a b + c d

inorder : (a + b) ∗ (c + d)

postorder : a b + c d + ∗

Note: For inorder we have to use paranteses in order to perform the operations inright order !

LAD, complexity 21

Stacks

Compute expressions on postfix form

Given a sequens of symbols, in postfix form, as an iterator it and a stack st

As long as it not emptylet sym be next symbol in it

if sym is an operandput sym onto st

else: if the operator is +Get the two uppermost operands from st,add them and put the result onto st

else: Get the two uppermost operands from st ,multiply them and put the result onto st

The result is now on top of the stack st !

LAD, complexity 22

Stacks

Compute expressions on postfix form, Java

Note: In most implementations pop gives the removed object as a result.

Iterator<String> it = ...;

Stack<Integer> st = new Stack<Integer>();

while ( it.hasNext() ) {

String sym = it.next();

switch ( sym.charAt(0) ) {

case ’+’ : st.push( st.pop() + st.pop() ); break;

case ’*’ : st.push( st.pop() * st.pop() ); break;

default : push( Integer.parsInt( sym ) );

}

}

LAD, complexity 23

Stacks

Stack implementation

May be implemented efficiantly either by array or as a linked structure.

An array is allocated with expected maximum size of the stack, if possible, and a fieldkeeping track of the stack top. All operations will then be of O(1).

For the linked structure, a singly linked structure, is apropiate, since all operationsonly concern the first element in the list. Here also, the operations are of O(1).

LAD, complexity 24