Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T...

27
Lecture 3
  • date post

    23-Jan-2016
  • Category

    Documents

  • view

    218
  • download

    0

Transcript of Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T...

Page 1: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Lecture 3

Page 2: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

kf(n) is O(f(n)) for any positive constant k

f(n) + g(n) is O(f(n)) if g(n) is O(f(n))

T1(n) is O(f(n)), T2(n) is O(g(n))

T1(n) T2(n) is O(f(n)g(n))

T1(n) + T2(n) is O(max(f(n),g(n)))

nr is O(np) if r p

nr is O(exp(n))

log n is O (nr) if r 0

f(n) is O(g(n)), g(n) is O(h(n)), Is f(n) O(h(n)) ?

Is kn O(n2) ?

Page 3: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Algorithm Complexity Analysis

diff = sum = 0;

For (k=0: k < N; k++)

sumsum + 1;

diff diff - 1;

For (k=0: k < 3N; k++)

sumsum - 1;

Page 4: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

First line takes 2 basic steps

Every iteration of first loop takes 2 basic steps.

First loop runs N times

Every iteration of second loop takes 1 basic step

Second loop runs for 3N times

Overall, 2 + 2N + 3N steps

This is O(N)

Page 5: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

RulesComplexity of a loop:

O(Number of iterations in a loop * maximum complexity of each iteration)

Nested Loops:

Analyze the innermost loop first, complexity of next outer loop = number of iterations in this loop * complexity of inner loop, etc…..

sum = 0;

For (i=0; i < N; i++)

For (j=0; j < N; j++) sumsum + 1;

Page 6: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

If (Condition)

S1

Else S2

Maximum of the two

If (yes)

print(1,2,….1000N)

Else print(1,2,….N2)

Page 7: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Maximum Subsequence Problem

There is an array of N elements

Need to find i, j such that the sum of all elements between the ith and jth position is maximum for all such sums

Maxsum = 0;

For (i=0; i < N; i++)

For (j=i; j < N; j++)

{ Thissum = sum of all elements between ith and jth positions;

Maxsum = max(Thissum, Maxsum);}

Page 8: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Analysis

Inner loop:

j=iN-1

(j-i + 1) = (N – i + 1)(N-i)/2

Outer Loop:

i=0N-1 (N – i + 1)(N-i)/2 = (N3 + 3N2 + 2N)/6

Page 9: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Maxsum = 0;

For (i=0; i < N; i++)

For (Thissum=0;j=i; j < N; j++)

{ Thissum = Thissum + A[i];

Maxsum = max(Thissum, Maxsum);}

Complexity?

Page 10: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Divide and Conquer

Break a big problem into two small sub-problems

Solve each of them efficiently.

Combine the two solutions

Page 11: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Maximum subsequence sum by divide and conquer

Divide the array into two parts: left part, right part

Max. subsequence lies completely in left, or completely in right or spans the middle.

If it spans the middle, then it includes the max subsequence in the left ending at the last element and the max subsequence in the right starting from the center

Page 12: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

4 –3 5 –2 -1 2 6 -2

Max subsequence sum for first half = 6

second half = 8

Max subsequence sum for first half ending at the last element is 4

Max subsequence sum for sum second half starting at the first element is 7

Max subsequence sum spanning the middle is ?

Max subsequence spans the middle

Page 13: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Maxsubsum(A[], left, right)

{

if left = right, maxsum = max(A[left], 0);

Center = (left + right)/2

maxleftsum = maxsubsum(A[],left, center);

maxrightsum = maxsubsum(A[],center+1,right);

maxleftbordersum = 0;

leftbordersum = 0;

for (i=center; i>=left; i--)

leftbordersum+=A[i];

Maxleftbordersum=max(maxleftbordersum, leftbordersum);

Page 14: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Find maxrightbordersum…..

return(max(maxleftsum, maxrightsum, maxrightbordersum + maxleftbordersum);

Page 15: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Complexity Analysis

T(1)=1

T(n) = 2T(n/2) + cn

= 2.cn/2 + 4T(n/4) + cn

= 4T(n/4) + 2cn

= 8T(n/8) + 3cn

=…………..

= 2iT(n/2i) + icn

=………………… (reach a point when n = 2i i=log n

= n.T(1) + cnlog n

Page 16: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

n + cnlogn

= O(nlogn)

Page 17: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Master Theorem

T(1)=p

T(n) = aT(n/b) + cnk

Case (1): a bk then T(n) is O(nlogb

a )

Case(2): a = bk then T(n) is O(nk logn)

Case(3): a < bk then T(n) is O(nk )

Cormen, Leiserson, Rivest

Page 18: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Linear Complexity Algorithm

Maxsum = 0; Thissum = 0;

For (j=0; j<N; j++)

{

Thissum = Thissum + A[j];

If (Thissum 0), Thissum = 0;

If (Maxsum Thissum),

Maxsum = Thissum;

}

O(N) complexity

Page 19: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Binary Search

You have a sorted list of numbers

You need to search the list for the number

If the number exists find its position.

If the number does not exist you need to detect that

Page 20: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Search(num, A[],left, right)

{

if (left = right)

{

if (A[left ]=num) return(left) and exit;

else conclude NOT PRESENT and exit;

}

center = (left + right)/2;

If (A[center] num)

Search(num,A[],center + 1,right);

If (A[center]>num)

Search(num,A[],left,center );

If (A[center]=num) return(center) and exit;

}

Page 21: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Complexity Analysis

T(n) = T(n/2) + c

O(log n) complexity

Page 22: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Other Complexity Notation

Intuitively, (not exactly) f(n) is O(g(n)) means f(n) g(n)

g(n) is an upper bound for f(n).

Now a lower bound notation, (n)

f(n) is (g(n)) if f(n) cg(n) for some positive constant c, and all large n.

Limn (f(n)/g(n)) > 0

Page 23: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

f(n) is (g(n)) if f(n) is O(g(n)) and (g(n))

(g(n)) is ``asymptotic equality’’

Limn (f(n)/g(n)) is a positive constant

f(n) is o(g(n)) if f(n) is O(g(n)) but not (g(n))

``asymptotic strict inequality’’

f(n) is o(g(n)) if given any positive constant c, there exists some m such that f(n) cg(n) for all n m

If limn (f(n)/g(n)) = 0, then f(n) is o(g(n))

Page 24: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Asymptotically less than or equal to O

Asymptotically greater than or equal to

Asymptotically equal to

Asymptotically strictly less o

Page 25: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Examples

Page 26: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Implication of the Notation

Suppose, an algorithm has complexity (f(n)) . This means that there exists a positive constant c such that for all sufficiently large n, there exists at least one input for which the algorithm consumes at least cf(n) steps.

Page 27: Lecture 3. kf(n) is O(f(n)) for any positive constant k f(n) + g(n) is O(f(n)) if g(n) is O(f(n)) T 1 (n) is O(f(n)), T 2 (n) is O(g(n)) T 1 (n) T 2 (n)

Complexity of a Problem Vs Algorithm

A problem is O(f(n)) means there is some O(f(n)) algorithm to solve the problem.

A problem is (f(n)) means every algorithm that can solve the problem is (f(n))