Mathematical description
We want to compare the running time of one program to another
We want a mathematical description with the following characteristics: Worst case
We care mostly about how bad things could be
AsymptoticWe focus on the behavior as the input size gets larger and larger
Big Oh notation
Enter Big Oh notation Big Oh simplifies a complicated running
time function into a simple statement about its worst case growth rate
All constant coefficients are ignored All low order terms are ignored 3n + 3 is O(n) Big Oh is a statement that a particular
running time is no worse than some function, with appropriate constants
Big Oh simplification examples 147n3 + 2n2 + 5n + 12083 is
O(n3)
n1000 + 2n isO(2n)
15n2 + 6n + 7log n + 145 isO(n2)
659n + nlog n + 87829 isO(n log n)
Note: In CS, we use log2 unless stated otherwise
Mulitiplication by hand
How long does it take to do multiplication by hand?
123
x 456
738
615
492__
56088
Let’s assume that the length of the numbers is n digits (n multiplications + n carries) x n digits + (n + 1
digits) x n additions Running time: O(n2)
Finding the largest element in an array
How do we find the largest element in an array?
Running time: O(n) if n is the length of the array What if the array is sorted in ascending order?
Running time: O(1)
int largest = array[0];
for( int i = 1; i < array.length; i++ )if( array[i] > largest )
largest = array[i];
System.out.println("Largest: " + largest);
System.out.println("Largest: " + array[array.length-1]);
Bubble sort
Here is some code that sorts an array in ascending order
What is its running time?
Running time: O(n2)
for( int i = 0; i < array.length; i++ )for( int j = 0; j < array.length - 1; j++ )
if( array[j] > array[j + 1] ){
int temp = array[j];array[j] = array[j + 1];
array[j + 1] = temp; }
Hierarchy of complexities Here is a table of several different complexity
measures, in ascending order, with their functions evaluated at n = 100
Description Big Oh f(100)
Constant O(1) 1
Logarithmic O(log n) 6.64
Linear O(n) 100
Linearithmic O(n log n) 664.39
Quadratic O(n2) 10000
Cubic O(n3) 1000000
Exponential O(2n) 1.27 x 1030
Factorial O(n!) 9.33 x 10157
Practical implications
Computers get faster, but not in unlimited ways If computers get 10 times faster, here is how much a
problem from each class could grow and still be solvable
Description Big Oh Increase in Size
Constant O(1) Unlimited
Logarithmic O(log n) 1000
Linear O(n) 10
Linearithmic O(n log n) 10
Quadratic O(n2) 3-4
Cubic O(n3) 2-3
Exponential O(2n) Hardly changes
Factorial O(n!) Hardly changes
Rules of thumb
There is nothing better than constant time Logarithmic time means that the problem can
become much larger and only take a little longer Linear time means that time grows with the problem Linearithmic time is just a little worse than linear Quadratic time means that expanding the problem
size significantly could make it impractical Cubic time is about the reasonable maximum if we
expect the problem to grow Exponential and factorial time mean that we cannot
solve anything but the most trivial problem instances
Memory usage
Memory usage can be a problem If you run out of memory, your
program can crash Memory usage can have serious
performance consequences too
Memory
Remember, there are multiple levels of memory on a computer
Each next level is on the order of 500 times larger and 500 times slower
Cache•Actually on the CPU•Fast and expensive
RAM•Primary memory for a desktop computer•Pretty fast and relatively expensive
Hard Drive•Secondary memory for a desktop computer•Slow and cheap
1000X 1000X
1000X 1000X
Size Speed
Memory speed
If you can do a lot of number crunching without leaving cache, that will be very fast
If you have to fetch data from RAM, that will slow things down
If you have to read and write data to the hard drive (unavoidable with large pieces of data like digital movies), you will slow things down a lot
Measuring memory
Memory can be easier to estimate than running time
Depending on your input, you will allocate a certain number of objects, arrays, and primitive data types
It is possible to count the storage for each item allocated
Remember that a reference to an object or an array costs an additional 4 bytes on top of the size of the object
Memory footprints
Here are the sizes of various types in Java
Note that the N refers to the number of elements in the array or String
Type Bytes
boolean 1
char 2
int 4
double 8
Type Bytes
boolean[] 16 + N
char[] 16 + 2N
int[] 16 + 4N
double[] 16 + 8N
Type Bytes
reference 4
String 40 + 2N
object 8 + size of members
array of objects 16 + (4 + size of members)N
Searching for a number
Lets say that I give you a list of numbers, and I ask you, “Is 37 on this list?”
As a human, you have no problem answering this question, as long as the list is reasonably short
What if the list is an array, and I want you to write a Java program to find some number?
Search algorithm
Easy! We just look through every element in
the array until we find it or run out
If we find it, we return the index, otherwise we return -1
public static int find( int[] array, int number ) {for( int i = 0; i < array.length; i++ )
if( array[i] == number )return i;
return -1;}
How long does it take?
Unfortunately for you, we know about Big Oh notation
Now we have some way to measure how long this algorithm takes
How long, if n is the length of the array?
O(n) time because we have to look through every element in the array, in the worst case
Can we do better?
Is there any way to go smaller than O(n)?
What complexity classes even exist that are smaller than O(n)? O(1) O(log n)
Well, on average, we only need to check half the numbers, that’s ½ n which is still O(n)
Darn…
We can’t do better unless…
We can do better with more information
For example, if the list is sorted, then we can use that information somehow
How? We can play a High-Low game
Binary search
Repeatedly divide the search space in half
We’re looking for 37, let’s say
5423 31
Check the middle(Too high)
Check the middle(Too low)
Check the middle(Too low)
Check the middle(Found it!)
37
So, is that faster than linear search?
How long can it take? What if you never find what you’re
looking for? Well, then, you’ve narrowed it down
to a single spot in the array that doesn’t have what you want
And what’s the maximum amount of time that could have taken?
Running time for binary search
We cut the search space in half every time
At worst, we keep cutting n in half until we get 1
Let’s say x is the number of times we look:
The running time is O(log n)
Guessing game
We can apply this idea to a guessing game
First we tell the computer that we are going to pick a number between 1 and n
We pick, and it tries to narrow down the number
It should only take log n tries Remember log2(1,000,000) is only
about 20
Interview question
This is a classic interview question asked by Microsoft, Amazon, and similar companies
Imagine that you have 9 red balls One of them is just slightly heavier
than the others, but so slightly that you can’t feel it
You have a very accurate two pan balance you can use to compare balls
Find the heaviest ball in the smallest number of weighings
What’s the smallest possible number?
It’s got to be 8 or fewer We could easily test one ball against
every other ball
There must be some cleverer way to divide them up
Something that is related somehow to binary search
That’s it!
We can divide the balls in half each time
If those all balance, it must be the one we left out to begin with
Nope, we can do better
How? They key is that you can actually cut the number of balls
into three parts each time
We weigh 3 against 3, if they balance, then we know the 3 left out have the heavy ball
When it’s down to 3, weigh 1 against 1, again knowing that it’s the one left out that’s heavy if they balance
Thinking outside the box, er, ball
The cool thing is… Yes, this is “cool” in the CS sense, not in
the real sense Anyway, the cool thing is that we are
trisecting the search space each time
This means that it takes log3 n weighings to find the heaviest ball
We could do 27 balls in 3 weighings, 81 balls in 4 weighings, etc.
Top Related