Cs1311lecture23wdl
-
Upload
muhammad-wasif -
Category
Technology
-
view
49 -
download
0
Transcript of Cs1311lecture23wdl
![Page 1: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/1.jpg)
Algorithm Cost Algorithm Complexity
![Page 2: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/2.jpg)
Algorithm Cost
![Page 3: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/3.jpg)
Back to Bunnies
• Recall that we calculated Fibonacci Numbers using two different techniques
– Recursion
– Iteration
LB
![Page 4: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/4.jpg)
Back to Bunnies
• Recursive calculation of Fibonacci Numbers:
Fib(1) = 1
Fib(2) = 1
Fib(N) = Fib(N-1) + Fib(N-2)
So:
Fib(3) = Fib(2) + Fib(1)
= 1 + 1
= 2
LB
![Page 5: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/5.jpg)
Tree Recursion?
f(n)
f(n-1) f(n-2)
f(n-2) f(n-3) f(n-4)f(n-3)
f(n-3) f(n-4)
f(n-4) f(n-5)
f(n-4) f(n-5)
f(n-5) f(n-6)
LB
![Page 6: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/6.jpg)
Tree Recursion Example
f(6)
f(5) f(4)
f(4) f(3) f(2)f(3)
f(3) f(2)
f(2) f(1)
f(2) f(1)
f(2) f(1)
LB
![Page 7: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/7.jpg)
Recursively
public static int fibR(int n)
{
if(n == 1 || n ==2)
return 1;
else
return fibR(n-1) + fibR(n-2);
}
LB
![Page 8: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/8.jpg)
Iteratively
public static int fibI(int n)
{
int oldest = 1;
int old = 1;
int fib = 1;
while(n-- > 2) {
fib = old + oldest;
oldest = old;
old = fib;
}
return fib;
}
LB
![Page 9: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/9.jpg)
Slight ModificationsLB
public static int fibR(int n)
{
if(n == 1 || n ==2)
return 1;
else
return fibR(n-1) + fibR(n-2);
}
public static int fibR(int n)
{
if(n == 1 || n ==2)
return 1;
else
return fibR(n-1) + fibR(n-2);
}
public static int fibI(int n)
{
int oldest = 1;
int old = 1;
int fib = 1;
while(n-- > 2) {
fib = old + oldest;
oldest = old;
old = fib;
}
return fib;
}
public static int fibI(int n)
{
int oldest = 1;
int old = 1;
int fib = 1;
while(n-- > 2) {
fib = old + oldest;
oldest = old;
old = fib;
}
return fib;
}
AddCounters
![Page 10: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/10.jpg)
Demo
LB
![Page 11: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/11.jpg)
Conclusion
Algorithm choice or design can make a big difference!
LB
![Page 12: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/12.jpg)
Correctness is Not Enough
• It isn’t sufficient that our algorithms perform the required tasks.
• We want them to do so efficiently, making the best use of– Space– Time
![Page 13: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/13.jpg)
Time and Space
• Time– Instructions take time.– How fast does the algorithm perform?– What affects its runtime?
• Space– Data structures take space.– What kind of data structures can be used?– How does the choice of data structure affect
the runtime?
![Page 14: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/14.jpg)
Time vs. Space
Very often, we can trade space for time:
For example: maintain a collection of students’ with SSN information.– Use an array of a billion elements and have
immediate access (better time)– Use an array of 35 elements and have to
search (better space)
![Page 15: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/15.jpg)
The Right Balance
The best solution uses a reasonable mix of space and time.
– Select effective data structures to represent your data model.
– Utilize efficient methods on these data structures.
![Page 16: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/16.jpg)
Questions?
![Page 17: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/17.jpg)
Algorithm Complexity
![Page 18: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/18.jpg)
Scenarios
• I’ve got two algorithms that accomplish the same task– Which is better?
• Given an algorithm, can I determine how long it will take to run?– Input is unknown– Don’t want to trace all possible paths of
execution
• For different input, can I determine how an algorithm’s runtime changes?
![Page 19: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/19.jpg)
Measuring the Growth of Work
While it is possible to measure the work done by an algorithm for a given set of input, we need a way to:
– Measure the rate of growth of an algorithm based upon the size of the input
– Compare algorithms to determine which is better for the situation
![Page 20: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/20.jpg)
Introducing Big O
• Will allow us to evaluate algorithms.• Has precise mathematical definition• We will use simplified version in CS 1311
• Caution for the real world: Only tells part of the story!
• Used in a sense to put algorithms into families
LB
![Page 21: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/21.jpg)
Why Use Big-O Notation
• Used when we only know the asymptotic upper bound.
• If you are not guaranteed certain input, then it is a valid upper bound that even the worst-case input will be below.
• May often be determined by inspection of an algorithm.
• Thus we don’t have to do a proof!
![Page 22: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/22.jpg)
Size of Input
• In analyzing rate of growth based upon size of input, we’ll use a variable– For each factor in the size, use a new variable– N is most common…
Examples:– A linked list of N elements– A 2D array of N x M elements– A Binary Search Tree of P elements
![Page 23: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/23.jpg)
Formal Definition
For a given function g(n), O(g(n)) is defined to be the set of functions
O(g(n)) = {f(n) : there exist positive constants c and n0 such that 0 f(n) cg(n) for all n n0}
![Page 24: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/24.jpg)
Visual O() Meaning
f(n)
cg(n)
n0
f(n) = O(g(n))
Size of input
Wo
rk d
on
e
Our Algorithm
Upper Bound
![Page 25: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/25.jpg)
Simplifying O() Answers(Throw-Away Math!)
We say 3n2 + 2 = O(n2) drop constants!
because we can show that there is a n0 and a c such that:
0 3n2 + 2 cn2 for n n0
i.e. c = 4 and n0 = 2 yields:
0 3n2 + 2 4n2 for n 2
![Page 26: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/26.jpg)
Correct but Meaningless
You could say
3n2 + 2 = O(n6) or 3n2 + 2 = O(n7)
But this is like answering:• What’s the world record for the mile?
– Less than 3 days.• How long does it take to drive to Chicago?
– Less than 11 years.
![Page 27: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/27.jpg)
Comparing Algorithms
• Now that we know the formal definition of O() notation (and what it means)…
• If we can determine the O() of algorithms…• This establishes the worst they perform.
• Thus now we can compare them and see which has the “better” performance.
![Page 28: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/28.jpg)
Comparing Factors
N
log N
N2
1
Size of input
Wo
rk d
on
e
![Page 29: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/29.jpg)
Correctly Interpreting O()
O(1) or “Order One”– Does not mean that it takes only one operation
– Does mean that the work doesn’t change as N
changes– Is notation for “constant work”
O(N) or “Order N”– Does not mean that it takes N operations– Does mean that the work changes in a way
that is proportional to N– Is a notation for “work grows at a linear rate”
![Page 30: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/30.jpg)
Complex/Combined Factors
• Algorithms typically consist of a sequence of logical steps/sections
• We need a way to analyze these more complex algorithms…
• It’s easy – analyze the sections and then combine them!
![Page 31: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/31.jpg)
Example: Insert in a Sorted Linked List
• Insert an element into an ordered list…– Find the right location– Do the steps to create the node and add it to
the list
17 38 142head //
Inserting 75
Step 1: find the location = O(N)
![Page 32: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/32.jpg)
Example: Insert in a Sorted Linked List
• Insert an element into an ordered list…– Find the right location– Do the steps to create the node and add it to
the list
17 38 142head //
Step 2: Do the node insertion = O(1)
75
![Page 33: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/33.jpg)
Combine the Analysis
• Find the right location = O(N)• Insert Node = O(1)
• Sequential, so add:– O(N) + O(1) = O(N + 1) =
Only keep dominant factor
O(N)
![Page 34: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/34.jpg)
Example: Search a 2D Array• Search an unsorted 2D array (row, then column)
– Traverse all rows– For each row, examine all the cells (changing columns)
Row
Column
12345
1 2 3 4 5 6 7 8 9 10
O(N)
![Page 35: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/35.jpg)
Example: Search a 2D Array• Search an unsorted 2D array (row, then column)
– Traverse all rows– For each row, examine all the cells (changing columns)
Row
Column
12345
1 2 3 4 5 6 7 8 9 10
O(M)
![Page 36: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/36.jpg)
Combine the Analysis
• Traverse rows = O(N)– Examine all cells in row = O(M)
• Embedded, so multiply:– O(N) x O(M) = O(N*M)
![Page 37: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/37.jpg)
Sequential Steps
• If steps appear sequentially (one after another), then add their respective O().
loop
. . .
endloop
loop
. . .
endloop
N
M
O(N + M)
![Page 38: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/38.jpg)
Embedded Steps
• If steps appear embedded (one inside another), then multiply their respective O().
loop
loop
. . .
endloop
endloop
M N O(N*M)
![Page 39: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/39.jpg)
Correctly Determining O()
• Can have multiple factors:– O(N*M)– O(logP + N2)
• But keep only the dominant factors:
– O(N + NlogN)
– O(N*M + P)
– O(V2 + VlogV)
• Drop constants:
– O(2N + 3N2)
O(NlogN)
remains the same
O(V2)
O(N2)O(N + N2)
![Page 40: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/40.jpg)
Summary
• We use O() notation to discuss the rate at which the work of an algorithm grows with respect to the size of the input.
• O() is an upper bound, so only keep dominant terms and drop constants
![Page 41: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/41.jpg)
Questions?
![Page 42: Cs1311lecture23wdl](https://reader036.fdocuments.in/reader036/viewer/2022081519/5563a3cdd8b42aae0d8b4d2a/html5/thumbnails/42.jpg)