1

Chapter 1

Analysis Basics

2

Chapter Outline

• What is analysis?

• What to count and consider

• Mathematical background

• Rates of growth

• Tournament method

• Analyzing programs

3

Prerequisites

• Before beginning this chapter, you should be able to:– Read and create algorithms– Identify comparison and arithmetic operations– Use basic algebra

4

Goals

• At the end of this chapter you should be able to:– Describe how to analyze an algorithm– Explain how to choose the operations that are

counted and why others are not– Explain how to do a best-case, worst-case,

and average case analysis

5

Goals (continued)

– Work with logarithms, probabilities, and summations

– Describe (f), Ω(f), O(f), growth rate, and algorithm order

– Use a decision tree to determine a lower bound on complexity

6

Analyzing an Algorithm

• Show that it properly solves the problem• Determine how efficient it solves the problem

– Approximate the number of operation

7

Algorithm Classification

• Iterative– Based on loop and conditional statements– Determine the work in the loop and the number of

times the loop executes

• Recursive– Based on calls to the same subprogram with a

smaller problem– Determine the number of smaller problems and

the amount of work done to produce them and then put their solutions together

8

What is Analysis?

• An approximation of how long an algorithm will take or how much extra space it needs

• Comparison of two or more algorithms that solve the same problem

• Independent of the computer because a faster computer does not make an algorithm more efficient

• Independent of initializations because those may be done faster

9

Input Classes

• Groups of input sets that cause the same amount of work to be done

• For example, finding the largest value– Input classes could be based on the location of the

largest element

10

Space Complexity

• The amount of space needed for an algorithm to complete its task

• Algorithms are classified as either in place or needing extra space– Depends on whether the algorithm moves data

around within its current storage or copies information to a new space while it’s working

• Use of extra space can be critical in software designed for small embedded systems

11

What to Count

• Comparisons– Equal, greater, not equal, …

• Arithmetic– Additions

• add, subtract, increment, decrement– Multiplications

• multiply, divide, modulus, remainder

12

Cases to Consider

• Best Case– The least amount of work done for any input set

• Worst Case– The most amount of work done for any input set

• Average Case– The amount of work done averaged over all of the

possible input sets

13

Average Case

• Determining the average case:– Find the number of input set classes– Find the probability that the input will be from each

of these classes– Find the amount of work done for each class

• The average case is given by:

m

iii tpn

1* )(A

14

Mathematical Background

• The floor of X is the largest integer to X• The ceiling of X is the smallest integer to X• The logarithm (base Y) of X is the value Z

that satisfies the equation X = YZ

15

Binary Trees

• A binary tree with N nodes has at least lg N+1 levels

• There are 2K-1 nodes on level K• In a complete binary tree all of the nodes on

levels 1 to J-1 have two children, so all of the leaves are on level J

• A complete binary tree with J levels has 2J-1 nodes

16

Probabilities

• A probability is a number between 0 and 1• If something will never occur, it has a

probability of 0• If something always occurs, it has a

probability of 1• If there are N equally likely events, each one

has a probability of 1/N

17

Summations

• A summation is a compact way to express the sum of a series of values

• The sum of the numbers from 1 to 7 would be written:

• There are closed forms for many summations given in the text

ii1

7

18

Rate of Growth

• The rate of growth of a function determines how fast the function value increases when the input increases

19

Rate of Growth• If algorithm A does x3 operations on an input of size

x and algorithm B does x2 operations, algorithm B is more efficient

• Because of the relative growth rate, we will consider x3 + x2 + x equivalent to x3

20

Classification of Growth

• Big Omega Ω(f)– The class of functions that grow at least as fast as

the function f, and maybe faster

• Big Oh O(f)– The class of functions that grow no faster than f,

and maybe slower

• Big Theta Θ(f)– The class of functions that grow at the same rate

as the function f

21

Tournament Method

• Finding the second largest element– Build a binary tree with the values in the leaves– The larger of a pair is copied into the parent node– The root has the largest value– The second largest value must have lost to the largest

22

Decision Trees

• Each choice is represented by a different child

• Path from the root to a leaf indicates how that solution is found

• Different algorithms that solve a problem will produce different decision trees

23

Decision Tree Example

24

Lower Bounds

• The absolute smallest number of operations needed to solve a problem

• Based on the problem not a particular algorithm

• Any algorithm that claims to solve the problem faster must fail

25

Lower Bounds Example

• For example, sorting a list– Each leaf holds one of the N! possible initial

orderings of elements– Each internal node indicates an element

comparison– If the tree is packed as tightly as possible, the

number of levels indicate the lower bound– Thus, the lower bound for sorting is N lg N

26

Analyzing Programs

• Program profilers are available that will indicate where the program is spending its execution time

• Can be done by hand by:– Incrementing a unique counter every time each

subprogram is called– Incrementing a unique counter every time a

particular code segment is reached