CS 231 Data Structures and AlgorithmsFall 2018

Algorithm AnalysisLecture 16 – October 10, 2018

Prof. Zadia Codabux

1

Agenda

• Algorithm Analysis

2

Administrative

• No quiz this week

3

Algorithm Analysis

4

Algorithm Analysis

• An algorithm is a sequence of instructions or a set of rules that are followed to accomplish a task. • e.g. making coffee requires a series of step

• To accomplish a task, there are more than one ways (algorithms). E.g., you can use K-cup coffee maker or French press pot. Some solutions are more efficient, taking less resources than others.

• How can we tell which solutions are more efficient?• How fast they run / how long they take to execute (time complexity). • How much memory or space they consume (space complexity).

We will focus on time complexity. Time complexity analysis is a simplified mathematical way of analyzing how long an algorithm with a given number of input(s) will take to complete it’s task. It is usually defined using Big-O notation.

5

Algorithm Analysis

• Recall the grocery checkout example

• If you were to choose a data structure to store the prices of items. At checkout, when the cashier scans the barcode of an item, the price is retrieved. • You have a choice between different data structures. How do you decide

which is the best for the job? What is defined as “better” or “best”?

6

Phonebook Example

• Scenario 1: Look up business phone number given page number and business name.

• Scenario 2: Given a phone number, find the person or business with that number.

• Scenario 3: Let n indicate the number of people in the phone book. Let's say everyone in the phone book gets a copy of the book. Let's say there is a numeric error in each person's number in each copy of the book.

7

Big-Oh Notation

We use Big-O notation to describe the order of operations that an algorithm will take to run as the size of problem (n) increases.

For example, iterating through a linked list will take time proportional to n. Suppose it took T(n) = 3.25n + 4.2 milliseconds.

To use Big-O notation, we ignore the constants that don’t have that big impact on the overall complexity of the problem. (Note that this does not mean the constants are unimportant for actual runtime or efficiency!)

8

Big-Oh Notation

Formally, Wikipedia puts it like this

9So our function T(n) = 3.25n + 42 is O(n).

Big-Oh Notation

• Likewise, if we have another algorithm whose runtime follows a function more like this: T(n) = 5n2 + 23n - 80000, we examine each term and use the term that grows the fastest as n grows. That is 5n2. Then, we ignore that constant and say that this second algorithm is O(n2).

• In short, we look at the term in our expression that has the potential greatest impact on the value that our expression will return. (This changes as the constant gets extremely large and n gets small, but let’s not worry about about that for now.)

• There are several common time complexities with simple examples.

10

Constant Time: O(1)

The number of steps the algorithm takes to accomplish the task is not proportional to the input size n.

int[] ary = {1,2,3,4,5,6,7,8,9};int get (int index) {

return ary[index];}

Other examples: adding two numbers, assigning a value to a variable, comparing two numbers

11

Logarithmic Time: O(log n)

The amount of work the algorithm needs to do decreases at each step (binary search in a sorted array)

int[] ary = {1,2,3,4,5,6,7,8,9};

boolean search (int num, int start, int end, int[] ary) {int midPos = (end-start)/2 + 1;if (end < start) return false;if (ary[midPos] == num) return true;else if (ary[midPos] > num) {

return search(num, start, midPos-1, ary); }else {

return search(num, midPos+1, end, ary); }}

search(3, 0, ary.length-1, ary);

12

Linear Time: O(n)

The number of steps the algorithm needs to accomplish the task is proportional to the input size n.

int[] a = {1,2,3,4,5,6,7,8,9};for (int i = 0; i < a.length; i++) {

System.out.println(a[i]);}

13

Here, n=a.length

• 1 initialization of I (constant)• n comparisons of i against a.length• n increments of i• n array indexing operations (to compute a[i])• n invocations of System.out.println

T(n)= 4n+1.

All operations are not created equal. But overall, T(n)= n.

Linear Time: O(n)

Imagine a case with doubly-nested loops where only the outer loop is dependent on the problem size n, and the inner loop always executes a constant number of times, say 3 times:

for ( int i = 0; i < n; i++ ) { for ( int j = 0; j < 3; j++ ) {// these statements are executed O(n) times }

}

Time Complexity: n

14

Quadratic Time: O(n2)

// print out the 2D array in a gridfor (int r = 0; r < n; r++) {

for (int c = 0; c < n; c++)

System.out.print(matrix[r][c]);}

15

Time complexity of the nested for loop

• 1 initialization of r (constant)• n initialization of c• n comparisons of r against matrix.length• n x n comparisons of c against

matrix[0].length• n increments of r• n x n increments of c• n x n array indexing operations (to compute

matrix[r][c])• n x n invocations of System.out.println• T(n) = 4 x n x n + 3 x n + 1 = 4n2 + 3n + 1

Overall, T(n) = O(n2)

Cubic Time: O(n3)

for ( int i = 0; i < n; i++ ) { for ( int j = 0; j < n; j++ ) {

for ( int k = 0; k < n; k++ ) {// these statements are executed O(n3) times

} }

}

16

Exponential Time: O(cn)

• Given an input of size n, the number of steps the algorithm takes to accomplish a task is a constant to the n power.

Example: trying to find every combination of letters for a password of length n.

17

Algorithm Analysis

18Source: Wikipedia

Best, Worst and Average Case

• Worst-case - An upper bound on the running time for any input of given size• Average-case - Assume all inputs of a given size are equally likely• Best-case - The lower bound on the running time

Complexity of algorithms is usually evaluated in the worst case (most unfavorable scenario). This means in the average case they can work faster, but in the worst case they work with the evaluated complexity and not slower.

19

Best, Worst and Average Case

Example: Sequential search in a list of size n

To find the searched key in the worst case, we have to check all the elements in the array. Worst-case: n comparisons O(N)

In the best case we will have luck and we will find the element at first position. Best-case: 1 comparison O(1)

In the average case we can expect to check half the elements in the array until we find the one we are looking for. Average-case: n/2 comparisons O(N/2) = O(N) – ignore the constants

20

Time Complexity of Data Structures

Array/ ArrayList• Get (using index) : O(1)• Search (a value) : O(n)• Add/Remove (at end) : O(1)• Add/Remove (at end) when full: O(n)

Singly Linked ListAddFirst/ RemoveFirst : O(1)Traversing and Add/Remove (at index) : O(n)Get (using index) : O(n)

21

Time Complexity of Data Structures

Array-based/ Node-based Stack• Push : O(1)• Pop : O(1)• Peek : O(1)

22