Faculty of Communications, Health and Science School of Computer and Information Science CSP1250...

Java Collections by D Watt & D Brown (2001) Edited and Modified by Dr W Guo (2003) 1

Faculty of Communications, Health and ScienceSchool of Computer and Information Science

CSP1250 Data Structures with Java

2. Algorithms Analysis

Fundamental Principles of AlgorithmsTime and Space Efficiency of AlgorithmsComplexity of AlgorithmsO-notation.Recursion.Guide.




Principles (1)An algorithm is a step-by-step procedure for solving a stated problem.The algorithm will be performed by a processor (which may be human, mechanical, or electronic).The algorithm must be expressed in steps that the processor is capable of performing.The algorithm must eventually terminate.




Principles (2)The algorithm must be expressed in some language that the processor “understands”. (But the underlying procedure is independent of the particular language chosen.)The stated problem must be solvable, i.e., capable of solution by a step-by-step procedure.




EfficiencyGiven several algorithms to solve the same problem, which algorithm is “best”?Given an algorithm, is it feasible to use it at all? In other words, is it efficient enough to be usable in practice?How much time does the algorithm require?How much space (memory) does the algorithm require?In general, both time and space requirements depend on the algorithm’s input (typically the “size” of the input).




Example: efficiencyHypothetical profile of two sorting algorithms:

• Algorithm B’s time grows more slowly than A’s.items to be sorted, n 10

1

2

3

time (ms)

Key:

Algorithm A

Algorithm B

0

4

20 30 40




Efficiency: measuring timeMeasure time in seconds?+ is useful in practice– depends on language, compiler, and processor.

Count algorithm steps?+ does not depend on compiler or processor– depends on granularity of steps.

Count characteristic operations? (e.g., arithmetic ops in math algorithms, comparisons in searching algorithms)+ depends only on the algorithm itself+ measures the algorithm’s intrinsic efficiency.




Example: power algorithms (1)

Simple power algorithm:To compute bn:

1. Set p to 1.2. For i = 1, …, n, repeat:

2.1. Multiply p by b.3. Terminate with answer p.





Analysis (counting multiplications):Step 2.1 performs a multiplication.This step is repeated n times.No. of multiplications = n




Example: power algorithms (3)Implementation in Java:

public class power{static int power (int b, int n){// Return b^n (where n is non-negative

int p = 1;for (int i=1; i<=n;i++) p *= b; return p;}

public static void main (String [] args){int p = Integer.parseInt (args [0]);System.out.println ("The power of b^n is " + p);}}




Example: power algorithms (4)Idea: b1000 = b500 b500. If we know b500, we can compute b1000 with only 1 more multiplication!Smart power algorithm:To compute bn:1. Set p to 1, set q to b, and set m to n.2. While m > 0, repeat:

2.1. If m is odd, multiply p by q.2.2. Halve m (discarding any remainder). 2.3. Multiply q by itself.

3. Terminate with answer p.





Analysis (counting multiplications):Steps 2.1–3 together perform at most 2 multiplications.They are repeated as often as we must halve the value of n (discarding any remainder) until it reaches 0, i.e., floor(log2 n) + 1 times.Max. no. of multiplications = 2(floor(log2 n) + 1)

= 2 floor(log2 n) + 2




Example: power algorithms (6)public class power1{static int power (int b, int n){// Return b^n (where n is non-negative) int p=1, q=b, m=n; while (m>0) { if (m%2 !=0) p *=q; m /=2; q *= q; } return p; }public static void main (String [] args){int p = Integer.parseInt (args [0]);System.out.println ("The power of b^n is " + p);}}




10 20 30 40

multiplications

10

20

30

40

50

5000

n


Comparison: simple power algorithm

smart power algorithm




ComplexityFor many interesting algorithms, the exact number of operations is too difficult to analyse mathematically.To simplify the analysis:– identify the fastest-growing term– neglect slower-growing terms– neglect the constant factor in the fastest-growing term.

The resulting formula is the algorithm’s time complexity. It focuses on the growth rate of the algorithm’s time requirement.Similarly for space complexity.




Example : analysis of power algorithms (1)

Analysis of simple power algorithm(counting multiplications):No. of multiplications = nTime taken is approximately proportional to n.Time complexity is of order n. This is written O(n).




Analysis of smart power algorithm (counting multiplic’ns):Max. no. of multiplications =

2 floor(log2 n) + 2

then to log2 n

Time complexity is of order log n.This is written O(log n).

then to floor(log2 n)

Simplify to 2 floor(log2 n)

Neglect slow-growing term, +2.

Neglect constant factor, 2.

Neglect floor(), which on average subtracts 0.5, a constant term.




10 20 30 40n

10

20

30

40

50

0

500

Comparison: n

log n





O-notation (1)

We have seen that an O(log n) algorithm is inherently better than an O(n) algorithm for large values of n. O(log n) signifies a slower growth rate than O(n).Complexity O(X) means “of order X”, i.e., growing proportionally to X.

Here X signifies the growth rate, neglecting slower-growing terms and constant factors.




O-notation (2)Common time complexities:O(1) constant time (feasible)O(log n) logarithmic time (feasible)O(n) linear time (feasible)O(n log n) log linear time (feasible)O(n2) quadratic time (sometimes feasible)O(n3) cubic time (sometimes feasible)O(2n) exponential time (rarely feasible)




Growth rates (1)Comparison of growth rates:

1 log n N nlog n n2 n3 n10 2n

1 1 0 1 0 1 1 1 2

2 1 1 2 2 4 8 1024 4

4 1 2 4 8 16 64 1048576 16

8 1 3 8 24 64 512 1.07E+09 256

10 1 3.321928 10 33.21928 100 1000 1E+10 1024

20 1 4.321928 20 86.43856 400 8000 1.02E+13 1048576

40 1 5.321928 40 212.8771 1600 64000 1.05E+16 1.1E+12

60 1 5.906891 60 354.4134 3600 216000 6.05E+17 1.15E+18

80 1 6.321928 80 505.7542 6400 512000 1.07E+19 1.21E+24

100 1 6.643856 100 664.3856 10000 1000000 1E+20 1.27E+30



10 20 30 40

20

40

60

0

80

100

500n

Growth rates (2)Graphically:

log n

n

n log nn22n



Example: growth rates (1)Consider a problem that requires n data items to be processed.Consider several competing algorithms to solve this problem. Suppose that their time requirements on a particular processor are as follows:

Algorithm Log: 0.3 log2 n secondsAlgorithm Lin: 0.1 n secondsAlgorithm LogLin: 0.03 n log2 n secondsAlgorithm Quad: 0.01 n2 secondsAlgorithm Cub: 0.001 n3 secondsAlgorithm Exp: 0.0001 2n seconds




Log

Lin

LogLin

Quad

Cub

Exp

10 20 30 40 50 60 70 80 90 1000

0:00

n

0:01

Example: growth rates (2)Compare how many data items (n) each algorithm can process in 1, 2, …, 10 seconds:

0:020:030:040:050:060:070:080:090:10




Recursion

A recursive algorithm is one expressed in terms of itself. In other words, at least one step of a recursive algorithm is a “call” to itself.In Java, a recursive method is one that calls itself.




When should recursion be used?

Sometimes an algorithm can be expressed using either iteration or recursion. The recursive version tends to be:+ more elegant and easier to understand– less efficient (extra calls consume time and space).

Sometimes an algorithm can be expressed only using recursion.




When does recursion work?Given a recursive algorithm, how can we sure that it terminates?The algorithm must have:– one or more “easy” cases– one or more “hard” cases.

In an “easy” case, the algorithm must give a direct answer without calling itself.In a “hard” case, the algorithm may call itself, but only to deal with an “easier” case.




Example: recursive power algorithms (1)Recursive definition of bn :

bn = 1if n = 0bn = b bn–1 if n > 0

Simple recursive power algorithm:To compute bn:1. If n = 0:

1.1. Terminate with answer 1.2. If n > 0:

2.1. Terminate with answer b bn–1.

Easy case: solved directly.

Hard case: solved by comput-ing bn–1, which is easier since n–1 is closer than n to 0.




Example: recursive power algorithms (2)

Implementation in Java: }





Idea: b1000 = b500 b500, and b1001 = b b500 b500.Alternative recursive definition of bn:

bn = 1 if n = 0bn = bn/2 bn/2 if n > 0 and n is evenbn = b bn/2 bn/2 if n > 0 and n is odd

(Recall: n/2 discards the remainder if n is odd.)





Smart recursive power algorithm:To compute bn:1. If n = 0:

1.1. Terminate with answer 1.2. If n > 0:

2.1. Let p be bn/2.2.2. If n is even:

2.2.1. Terminate with answer p p.2.3. If n is odd:

2.3.1. Terminate with answer b p p.

Easy case: solved directly.

Hard case: solved by comput-ing bn/2, which is easier since n/2 is closer than n to 0.




Example: recursive power algorithms (5)Implementation in Java:static int power (int b, int n) {// Return bn (where n is non-negative).

if (n == 0)return 1;

else {int p = power(b, n/2);if (n % 2 == 0) return p * p;else return b * p * p;

}}





Analysis (counting multiplications):

Each recursive power algorithm performs the same number of multiplications as the corresponding non-recursive algorithm. So their time complexities are the same:

non-recursive recursiveSimple power algorithm O(n) O(n)Smart power algorithm O(log n) O(log n)




Example: recursive power algorithms (7)Analysis (space):The non-recursive power algorithms use constant space, i.e., O(1).A recursive algorithm uses extra space for each recursive call. The simple recursive power algorithm calls itself n times before returning, whereas the smart recursive power algorithm calls itself floor(log2 n) times.

non-recursive recursiveSimple power algorithm O(1) O(n)Smart power algorithm O(1) O(log n)




Example: Towers of Hanoi (1)

Three vertical poles (1, 2, 3) are mounted on a platform.A number of differently-sized disks are threaded on to pole 1, forming a tower with the largest disk at the bottom and the smallest disk at the top. We may move one disk at a time, from any pole to any other pole, but we must never place a larger disk on top of a smaller disk.Problem: Move the tower of disks from pole 1 to pole 2.





Animation (with 2 disks):

1 2 31 2 31 2 31 2 3




Example: Towers of Hanoi (3)Towers of Hanoi algorithm:To move a tower of n disks from pole source to pole dest:

1. If n = 1:1.1. Move a single disk from source to dest.

2. If n > 1:2.1. Let spare be the remaining pole, other than source and dest.2.2. Move a tower of (n–1) disks from source to spare.2.3. Move a single disk from source to dest.2.4. Move a tower of (n–1) disks from spare to dest.

3. Terminate.





3. Terminate.

source dest



3. Terminate.

source dest spare

Example: Towers of Hanoi (4)Animation (with 6 disks):



3. Terminate.

source dest spare



3. Terminate.

source dest spare



3. Terminate.

source dest spare



3. Terminate.

source dest spare





Analysis (counting moves):Let the total no. of moves required to move a tower of n disks be moves(n). Then:

moves(n) = 1 if n = 1moves(n) = 1 + 2 moves(n–1)if n > 1

Solution:moves(n) = 2n – 1

Time complexity is O(2n).




Week 2 Study GuideReading:– Chapter 2: Java Collections by D.A. Watt and D.F. Brown (2001) – Appendix A: Java Collections by D.A. Watt and D.F. Brown (2001)

Workshop 2– Algorithm Analysis

Next week:– First Test on Algorithm Analysis– Lecture: Array Data Structures

Chapter 3: Java Collections by D.A. Watt and D.F. Brown (2001)

Faculty of Communications, Health and Science School of Computer and Information Science CSP1250...

Documents

Transcript of Faculty of Communications, Health and Science School of Computer and Information Science CSP1250...