Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II.
-
Upload
alexandra-rich -
Category
Documents
-
view
219 -
download
0
Transcript of Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II.
Asymptotic Complexity(Big-O Notation)
CS 1037 Fundamentals of Computer Science II
2
What is a “Fast” Algorithm?
Best case
Average case
Worst case
Small problems
Large problems
Best case
Average case
Worst case
Small problems
Large problems
Takes the fewest seconds? Takes the fewest steps?
In practice, these matter. Choice depends on: • application requirements• computer architecture• expected input
P PP P
In theoretical comp. sci., algorithms ranked by:• fewest # of steps,• in the worst case,• on large problems
T
3
Ranking Running-Time
• Suppose algorithm A takes an2+b steps on problem of size n in worst case
• Suppose algorithm B takes cnlgn+d steps• Which algorithm is faster in practice?– depends on a,b,c,d and typical input
• Which algorithm is faster in theory?– for large enough n coefficients a,b,c,d don’t matter– algorithm B therefore fundamentally faster than A
4
Example: Algorithm A vs B
• Force cnlgn+d to have large c...
B faster than A for large enough
nA
B
5
When is Algorithm A “as fast as” B?
• Let A(n) be # steps for A on problem size n• Let B(n) be # steps for algorithm B– e.g. define functions A(n)= 4n+2, B(n)= n2
A as fast as B if A(n) · kB(n) for all large enough n and some k > 0
4n+2 · n2 for all n ¸ 54n+2 · 2n2 for all n ¸ 320nlgn · n2 for all n ¸ 144
n
tim
e
# items 5
6
Asymptotic Notation (“Big-O”)
• Let g(n) be some function of n• Define O(g(n)) to be set of all functions
as fast as g(n) in theoretical sense:
O(g(n)) = f(n) j there exist positive k, n0
such that 0 · f(n) · kg(n) for all n ¸n0
f
g
n
tim
e
# items
n2
f(n) 2 O(n2)n0
n 2 O(n) n 2 O(n2)n2 2 O(n)
e.g.
7
Asymptotic Notation (“Big-O”)
• Intuition:
n+100 2 O(n) because n+100 · 50n for n ¸ 3 or because n+100 · 2n for n ¸ 100
n2 2 O(n) because for any k>0 eventually n2 > kn
for large n (i.e. “n2 slower than n”)
any constant c 2 O(n) because c · n for n ¸ c
f(n) 2 O(g(n))
“f(n) no slower than kg(n) for large n”
8
Asymptotic Notation (“Big-O”)
• Claim: any function of the form an+b with a ¸0 is a member of the set O(n)
Proof: choose k = a+j b j, then an+b · (a+j b j)n 8 n ¸ 1
• Claim: any an2+bn+c 2 O(n2) if a ¸0 Proof: choose k = a+j b j+j c j, then an2+bn+c · (a+j b j+j c j)n2 8 n ¸ 1
9
Asymptotic Notation (“Big-O”)
• Can say Insertion Sort runs in...– “quadratic time in the worst-case”, or– O(n2) time ... “oh of n squared time”
• Can say Binary Search runs in...– “logarithmic time in the worst-case”, or– O(lgn) time ... “oh of log n time”
• Can say Counting Sort runs in...– “time linear in n and k”, or– O(n+k) time ... “oh of n plus k time”
this k means “max range of items”
10
Asymptotic Notation (“Big-O”)
• Claim: if f(n) 2 O(g(n)) and g(n) 2
O(h(n)) then f(n) 2 O(h(n)) Proof: We know for some n1,n2,k1,k2 > 0
0·f(n)·k1g(n) 8 n ¸ n1
0·g(n)·k2h(n) 8 n ¸ n2
and therefore 0·f(n)·k1k2h(n) 8 n ¸ maxfn1,n2g
11
Big-O Notation Used Everywhere
DOCUMENTATION
COMP. SCI. COURSES
ALGORITHMS RESEARCH
12
More Examples of Big-O Membership
5050
n+506n2+50
n2+10n+50n3¡10n
2lgn+50n+lgn+50
2nlgn+n+lgn
2n+500n3
cn+nc
22222222222
O(1)O(n)O(n)O(n2)O(n2)O(n3)O(lgn)O(n)O(nlgn)O(2n)O(cn) if c >
1
We say any constant c 2 O(1), i.e. O(1) = set of all constants
Such functions are “exponential in n” and grow extremely fast! Such algorithms take longer than age of universe to compute medium-sized problems
13
History of Big-O
• Invented by mathematician Paul Bachmann in 1894 to characterize error bounds
• Don Knuth popularized use for running time bounds in 1970s
The Art of Computer Programming, Volume 1
14
Knuth on Importance of Big-O“Students will be motivated to use O(n) notation for two important reasons. First, it significantly simplifies calculations because it allows us to be sloppy — but in a satisfactorily controlled way. Second, it appears in the power series
calculations of symbolic algebra systems like Maple and Mathematica, which today’s students will surely be using...”
—Don Knuth advocating Big-O in high school, Letter to The American Mathematical Society March 1998
http://micromath.wordpress.com/2008/04/14/donald-knuth-calculus-via-o-notation/
15
END OF COURSE MATERIAL
hooray!!!
16
variablesif-elseloops
arrayspointersfunctions
CS1036
Programming Skills Take Time!eff
ectiv
enes
s of
you
stuff in your brain
structsclasses
templates
sortinglists/queues
trees
CS1037
dictionarieshashingmore…
CS2210SE2205
Big Universe of Ideas!
17
variablesloops if-else
arrayspointersfunctions
exceptions
structs
classes
templates
sortinglistsqueuestrees
big-O
recursion
polymorphism
inheritance
debuggingassertions
iterators
threads
networking
security
optimization
encapsulationsearching
hashing
clusteringheaps
balanced trees
caching
randomized algorithms
graphs
matching
NP-completeness
computability theory
finite automata
approximation algorithms
multisets
dynamic programming
compression
float arithmetic
symbolic computation
GPUs
distributed computing
compilers
operating systems
greedy algorithms
interpretersparsing
databases
dictionaries
stacks
user interfaces
JavaPython
virtual machinesC#
backtracking
profiling
reducibility
Cdata mining
assembly
file I/O
proof techniques
portabilitylinear algebra
rasterization
references
physics simulation
version control
concurrency
combinatorics
UML
19
Goals of CS 1037
• Understand how to…– use data structures– implement data structures– characterize an algorithm’s performance– express computation in C++
• Develop intuition about…– best data structure / algorithm for situation– good code, poor code, and dangerous code
• Programming experience, confidence
20
Pearls of Wisdom“Anybody who comes to you and says he has a perfect language is either naive or a salesman.” —Bjarne Stroustrup
at Waterloo Comp Sci Club Meeting
“A program that has not been tested does not work.” —Bjarne Stroustrup
21
Pearls of Wisdom“The psychological profiling [of a programmer] is mostly the ability to shift levels of abstraction, from low level to high level. To see something in the small and to see something in the large.” —Don Knuth
“Premature optimization is the root of all evil.”
—Don Knuth
remember that one!!
22
EXAM REVIEW TIME!
hooray?