1Computer Science — An Overview

J. Glenn Brookshear

Chapter FourAlgorithms

2

Online

• The Concept of an Algorithm

• Algorithm Representation

• Algorithm Discovery

• Search and Sorting

• Recursive Structures

• Efficiency and Correctness

3

Efficiency and Correctness

4

Efficiency of Algorithm

• The efficiency is an important issue of good algorithm.– How long does it take to find an entry by using

search algorithm?– Example: university registrar

• 30000 sorted records in the database, search one record by the student identification number

• Sequential search: average 15000 comparisons/search

• Binary search: maximum 15 comparisons/search

5

Analysis of Algorithm

• The purpose of algorithm analysis is to predict the behavior, specially the running time, of an algorithm without implementing it on a specific computer.

• We can compare different algorithms to determine the best one for our purposes.

• We wants a good way to measure speed which is independent of computer and programming language.

6

Time Complexity (1/2)

• The complexity of an algorithm is an estimate of the time taken, as a function of the amount n of information provided.

• There are too many influencing factors for the execution. Thus, we focus on the major steps only.– Example: For sorting algorithms, we count the

number of comparison and ignore the rest.

7

Time Complexity (2/2)

• We are interesting in the worst case, i.e. the upper bound of the running time.– We chose a special input to make the running time

is largest than those of other inputs. This is called as the worst case.

• Analysis consideration– Best case analysis– Worst case analysis– Average case analysis

8

Space Complexity

• The space complexity indicates the amount of temporary storage required for running the algorithm.

• The storage required for the input or for the output is ignore.

9

Steps in a for-loop

• The input size is n, then the number of this algorithm is n2+n.

• We only care the number of print (i,j) is n2.

for (j=0; j<n; j++) {

for (i=0; i<m; i++)

cout << i << “ and ” << j << “\n”;

cout << “***********\n”;

}

1 step for

n2 times1 step for n times

10

Steps in Sequential Search

• If List has n entries, then the number of this algorithm is 3n+4.

2 steps

1 step

1 step

3 steps for n times

1 step

11

Methodology of Complexity

• We need an methodology to denote the time (or space) complexity.

• The Big O notation is used to approximate the exact running time.– Example: sequential search has time

complexity O(n), where n is the size of list.– It means that “n steps is enough to find the

target among n items”.

The Big O Notation

•g(n) has the complexity of f(n).“g(n) has order of magnitude f(n)”

g(n) = O(f(n))

g(n) < c × f(n) for all large enough n

f(n) is an upper bound of g(n).– Example:

g(n)=5n2 + 10n + 25 = O(f(n)=n2) = O(n2)

•Define: g(n)=O(f(n)) iff c,N: constant g(n) cf(n), nN.

Why Approximate ?

•We want to ignore startup overheads.– 5n2+15 the running time is approximately n2

– If something takes 5n+15 steps, the overhead of 15 steps becomes insignificant for large n.

•We want to ignore constant factors. – 10n the running time is approximately n– Something which takes 10n steps on one computer might

only take 5n steps on another computer.

•We want to ignore the smaller one.– If something takes 2n2 + 5n steps, we ignore the 5n part,

because it is swamped by the 2n2 part for large n.

14

Approximate Approach

• Only concentrate on the size of input (n).

• We assume that n is large and ignore the redundancy.

• The complexity of an algorithm can often be estimated directly, rather than counting the number of steps and then simplifying.

• The analysis is only an approximation.

15

Example: g(n)=3n+4

• Find f(n)=n, c=4, N=4

g(n)=3n+4 4n, n4.

g(n)=O(f(n))=O(n)• The number of major steps in the sequential

search is bound by g(n)=3n+4.• The time complexity of the sequential search

is O(n). That is, we only care the number of comparisons in n.

16

Example: g(n)=5n2+15

• Let f(n)=n2, c=6, N=4

g(n)=5n2+15 6n2, n4.

g(n)=O(n2)

• Let f(n)=n3, c=1, N=6

g(n)=5n2+15 n3, n6.

g(n)=O(n3)

• Usually, we interested in the upper bound which is the closest to g(n).

17

Lemma

• f(n)=O(s(n)), g(n)=O(r(n))

f(n)+g(n)=O(s(n)+r(n))

• f(n)=O(s(n)), g(n)=O(r(n))

f(n)•g(n)=O(s(n)•r(n))

18

Example of Comparison

• Which algorithm is faster?– O(n) v.s. O(n1.8) v.s. O(n2)

• The numbers of three algorithms are 100n, 100n1.8 and 2n2+50, respectively.– n=5, 100n=500 > 2n2+50=100

However, n50, 100n < 2n2+50– n=1, 100n1.8=100 > 2n2+50=52

However, n3*108, 100n1.8 < 2n2+50

19

Notes of O Notation

• The approximate is valid if n is large enough.

• We ignore constant factors and concentrate on the behavior of the algorithm as the size of input goes to infinity.

• We say that the time complexity of an algorithm, or the the running time, is O(f(n)), if the number of major steps is bound by f(n).

20

Running Times (in Seconds)

• n=1000, the machine speed 1000steps/sec.

Running time Execution timelog2n 0.010

n log2n 1

n1.5 10

n2 32

n3 106

1.1n 1039

21

The and Notation

• Define: g(n)=Ω(f(n)) iff c,N: constant g(n) cf(n) nN.– f(n) is a lower bound of g(n).

• Define: g(n)=(f(n)) iff g(n)=O(f(n)) and g(n)=Ω(f(n)).– f(n) is a lower bound and an upper bound of

g(n).

22

Worst Case in Binary Search

• Input size: n

• The number of searches is log2n

23

Figure 4.20worst-case analysis of the binary search

algorithm: O(log2n)

24

Figure 4.18Applying the insertion sort in a worst-case

situation O(n2)

• The number of comparisons is 1+2+3+...+(n-1)=n(n-1)/2.

25

Figure 4.19Graph of the worst-case analysis of the

insertion sort algorithm O(n2)

26

Software Verification

• Any software needs verification.– Example: Gold chain with minimum cut

• We want a formal techniques to prove the correctness of a program.

27

Figure 4.21Separating the chain using only three cuts

28

Figure 4.22Solving the problem with only one cut

29

Hardware Verification

• The verification of circuit deigns and machine construction are important.– Mark I contained wiring errors.– The floating-point portion of the early Pentium

microprocessors.

30

Figure 4.23The assertions associated with a typical while

structure

31

Homework

• Problems: 14,15,16,41

• Social Issues: 1