CSC310 © Tom Briggs Shippensburg University Fundamentals of the Analysis of Algorithm Efficiency...

CSC310© Tom BriggsShippensburg University

Fundamentals of the Fundamentals of the Analysis of Analysis of

Algorithm EfficiencyAlgorithm Efficiency

Chapter 2Chapter 2


Analysis FrameworkAnalysis Framework

In what ways can we compare algorithms?In what ways can we compare algorithms? Time & Space efficiency are most commonTime & Space efficiency are most common Others: Cost, Power, …Others: Cost, Power, …

Characteristics of a good frameworkCharacteristics of a good framework Independence from Independence from

Programmer implementationProgrammer implementation Language / compiler optimizationsLanguage / compiler optimizations Architecture *Architecture *

Dependence onDependence on Input encodingInput encoding Critical operation of algorithmCritical operation of algorithm Frequency of critical operation executionFrequency of critical operation execution Number of “things” stored in memory relative to input encodingNumber of “things” stored in memory relative to input encoding


Input SizeInput Size

Goal: Goal: express efficiency of an algorithm relative to the size of its express efficiency of an algorithm relative to the size of its

encoded input.encoded input. We shall assume that all input to our algorithms is encoded We shall assume that all input to our algorithms is encoded

efficiently.efficiently. Example, integers will typically be encoded in binary, which Example, integers will typically be encoded in binary, which

implies the following:implies the following:

Suppose we don’t make this assertion?Suppose we don’t make this assertion?


Measuring Running TimeMeasuring Running Time

Basic OperationBasic Operation The operation contributing the most to total runtimeThe operation contributing the most to total runtime Frequency of execution depends on inputFrequency of execution depends on input

Let cLet c_op_op be the execution time of an algorithms basic be the execution time of an algorithms basic

operation on a particular computer, and let operation on a particular computer, and let C(n)C(n) be the be the number of times this operation is run on input n, then we number of times this operation is run on input n, then we can estimate the running time can estimate the running time T(n)T(n) of a program: of a program:

T(n)T(n) ¼¼ ccopop C(n)C(n)


Basic OperationBasic Operation

What is the basic operation for:What is the basic operation for: Given some set, S = { iGiven some set, S = { i11, i, i22, … i, … inn } of unordered integers, find the } of unordered integers, find the

minimumminimum Given two integers, a and b, find their sum, a + bGiven two integers, a and b, find their sum, a + b Given a set, S = { iGiven a set, S = { i11, i, i22, … i, … inn } and a, such that for all i } and a, such that for all ikk and a are and a are

all integers, determine whether S contains a.all integers, determine whether S contains a.


Unit Free ComparisonsUnit Free Comparisons

Goal: Goal: independence from architecture, implementation…independence from architecture, implementation…

Suppose we observe that some algorithm, A, makes Suppose we observe that some algorithm, A, makes C(n) basic operations for an input of size n, where C(n) C(n) basic operations for an input of size n, where C(n) is:is:

Suppose that each operation requires cSuppose that each operation requires c00 time T(n) time T(n) ¼¼ c c00 C(n). How C(n). How

much longer would the algorithm require for 2 times its input i.e: much longer would the algorithm require for 2 times its input i.e: T(2n) T(2n) ¼¼ ? ? Remember the speed-up formula?Remember the speed-up formula?


Orders of GrowthOrders of Growth


Big-Oh, Big-Big-Oh, Big-

OO-notation:-notation: A function t(n) is said to be in O(g(n)), denoted t A function t(n) is said to be in O(g(n)), denoted t 22 O(g(n)), if t(n) O(g(n)), if t(n)

is bounded above by some constant multiple of g(n) for all large is bounded above by some constant multiple of g(n) for all large n. n.

--notation:notation: A function t(n) is said to be in A function t(n) is said to be in (g(n)), denoted t (g(n)), denoted t 22 ( ((n)), if t(n) (n)), if t(n)

is bounded below by some constant multiple of g(n) for all large is bounded below by some constant multiple of g(n) for all large n.n.


Big-Big-

\Theta notation\Theta notation A function t(n) is said to be in \Theta(g(n)), denoted t(n) \in \A function t(n) is said to be in \Theta(g(n)), denoted t(n) \in \

Theta(g(n)), if t(n) is bounded both above and below by some Theta(g(n)), if t(n) is bounded both above and below by some positive constant multiples of g(n) for all large n, i.e. there must positive constant multiples of g(n) for all large n, i.e. there must exists some positive constant c_1 and c_2 and some non-exists some positive constant c_1 and c_2 and some non-negative integer, n_0, such that:negative integer, n_0, such that:


Example of Big-OhExample of Big-Oh

0 5 10 15 20 25-600

-400

-200

0

200

400

600

800Growth Class Comparison

y=1/2x2-100x

y=5x2-100x

t(n) <= c g(n), for all n > = 20


Asymptotic TightnessAsymptotic Tightness

Big-Oh does not have to be asymptotically tightBig-Oh does not have to be asymptotically tight t(n) = 0.5 n t(n) = 0.5 n 22 O(n) with c = 1, n O(n) with c = 1, n00 = 1, but is also in O(n = 1, but is also in O(n100100)…)…

Big-Big- isn’t tight either isn’t tight either t(n) = nt(n) = n55 22 (n) with c=1, n(n) with c=1, n00 = 1… = 1…

Big-Big- is tight… is tight… t(n) must be in same growth classes to meet definitiont(n) must be in same growth classes to meet definition Can you prove this assertion?Can you prove this assertion?


Prove Prove

Prove that t(n)=3nProve that t(n)=3n33+2n+2n22+1 +1 22 (n(n33).).

Show t(n) Show t(n) ·· c c11 g(n) for all n g(n) for all n ¸̧ n n00 : i.e. t(n) : i.e. t(n) 22 O(n O(n33))

And, show cAnd, show c00 g(n) g(n) ·· t(n) for all n t(n) for all n ¸̧ n n00 : i.e. t(n) : i.e. t(n) 22 (n(n33))


TheoremTheorem

TheoremTheorem

If tIf t11(n) (n) 22 O(g O(g11(n)) and t(n)) and t22(n) (n) 22 O(g O(g22(n)), then t(n)), then t11(n)+t(n)+t22(n) (n) 22

O(max{gO(max{g11(n), g(n), g22(n)})(n)})

Proof Sketch: Proof Sketch:

By defn, we can take max(cBy defn, we can take max(c11 and c and c22), and max(n), and max(n0101, and , and

nn0202))


General Plan for Non-Recursive Alg.General Plan for Non-Recursive Alg.

1.1. Decide on a parameter indicate input size Decide on a parameter indicate input size

2.2. Identify the algorithm’s basic operationIdentify the algorithm’s basic operation

3.3. Check whether the number of basic ops depends on Check whether the number of basic ops depends on input size input size

4.4. Set up a sum expressing the number of times the Set up a sum expressing the number of times the algorithm’s basic op is executed (see cheat sheet)algorithm’s basic op is executed (see cheat sheet)

5.5. Use standard formulae to compute closed-form formula Use standard formulae to compute closed-form formula for the count, and compute its order of growthfor the count, and compute its order of growth


ExampleExample

Algorithm: UniqueElementsAlgorithm: UniqueElements

// input A[0..n-1]// input A[0..n-1]

// output true if all elements of A are unique// output true if all elements of A are unique

for i for i ÃÃ 0 to n-2 do 0 to n-2 do

for j for j ÃÃ i +1 to n-1 do i +1 to n-1 do

if A[i] = A[j] return falseif A[i] = A[j] return false

return truereturn true


Analysis of Recursive AlgorithmsAnalysis of Recursive Algorithms

Recall:Recall:Recursive algorithm – Recursive algorithm –

Solves smaller problem of same typeSolves smaller problem of same typeHas a defined recursive case (calls self)Has a defined recursive case (calls self)Has a defined base case (trivial solution)Has a defined base case (trivial solution)All recursive cases eventually lead to a base caseAll recursive cases eventually lead to a base case

Frequently: no loops present (due to recursion)Frequently: no loops present (due to recursion)

Goal: Find and solve recurrence relations that express Goal: Find and solve recurrence relations that express run-time.run-time.


Example:Example:

What is the critical operation? What is the growth class?


Example (cont’d)Example (cont’d)

What is the critical operation here?

T(n)=?

Backward substitution: start with some n, and work backward, substituting values for the C(n), until a clear pattern emerges; then substitute for thebase case.


Another ExampleAnother Example

Tower of HanoiTower of Hanoi

Use backward substitution to solve thisUse backward substitution to solve this


Another ExampleAnother Example

What is the run-time if we assume that G is fully-connected, directed, weighted graph?

CSC310 © Tom Briggs Shippensburg University Fundamentals of the Analysis of Algorithm Efficiency...

Documents

Transcript of CSC310 © Tom Briggs Shippensburg University Fundamentals of the Analysis of Algorithm Efficiency...