Algorithm and Desing

7/31/2019 Algorithm and Desing

1/20

CHAPTER NO : 4

INTRODUCTION TO ALGORITHM


2/20


3/20

DEFINE RECURRENCE RELATION :

A recurrence relation for the sequence {an} is anequation that expresses an is terms of one or more ofthe previous terms of the sequence, namely, a0, a1,, a

n-1, for all integers n with

n n0, where n0 is a nonnegative integer.

A sequence is called a solution of a recurrencerelation if it terms satisfy the recurrence relation.


4/20

In other words, a recurrence relation is like arecursively defined sequence, but withoutspecifying any initial values (initial conditions).

Therefore, the same recurrence relation canhave (and usually has) multiple solutions.

Ifboth the initial conditions and the recurrence

relation are specified, then the sequence isuniquely determined.


5/20

Consider the recurrence relationan = 2an-1 an-2for n = 2, 3, 4,

Is the sequence {an} with an=3n a solution of this recurrence relation?

For n 2 we see that2an-1 an-2 = 2(3(n 1)) 3(n 2) = 3n = an.

Therefore, {an} with an=3n is a solution of the recurrence relation.

Is the sequence {an} with an=5 a solution of the same recurrence relation?

For n 2 we see that2an-1 an-2 = 25 - 5 = 5 = an.

Therefore, {an} with an=5 is also a solution of the recurrence relation.


6/20

There are four methods of recurrence

relation:

TREE METHOD.

ITREATION METHOD.

SUBSTITUTION METHOD.

MASTER METHOD .


7/20

Define Tree method :

Another common pattern of computation iscalled tree recursion. As an example, consider

computing the sequence of Fibonacci numbers, inwhich each number is the sum of the preceding two:

0,1,1,2,3,5,8,13,21,... In general, the Fibonacci numbers can be defined by

the rule

We can immediately translate this definition into arecursive procedure for computing Fibonaccinumbers:

(define (fib n) (cond ((= n 0) 0) ((= n 1) 1) (else (+(fib (- n 1)) (fib (- n 2))))))


8/20

T(n) = 2T(n/2) + n2.

The recursion tree for this recurrence has the

following form:


9/20

In this case, it is straightforward to sum

across each row of the tree to obtain the

total work done at a given level:


10/20

Define Iteration :

Iteration means the act of repeating a process

usually with the aim of approaching a desiredgoal or target or result. Each repetition of the

process is also called an "iteration," and the

results of one iteration are used as the starting

point for the next iteration.


11/20

T(n) = T((7/8)^1 * n) + 2 * (7/8)^0 * n =

T((7/8)^2 * n) + 2 * (7/8)^1 * n + 2 * (7/8)^0 * n

= T((7/8)^3 * n) + 2 * (7/8)^2 * n + 2 * (7/8)^1 *n + 2 * (7/8)^0 * n . . . = T((7/8)^k * n) + 2 * n *

sum j = 0 to k-1 (7/8)^j Now, let k tend to

infinity and see what happens. It would help if

you're familiar with geometric series.
http://en.wikipedia.org/wiki/Geometric_serieshttp://en.wikipedia.org/wiki/Geometric_series


12/20

Using Substitution Method :

The substitution method is a condensed way of provingan asymptotic bound on a recurrence by induction. Inthe substitution method, instead of trying to find an

exact closed-form solution, we only try to find a closed-form boundon the recurrence. This is often much easierthan finding a full closed-form solution, as there is muchgreater leeway in dealing with constants.

The substitution method is a powerful approach that is

able to prove upper bounds for almost all recurrences.However, its power is not always needed; for certaintypes of recurrences, the master method (see below)can be used to derive a tight bound with less work. Inthose cases, it is better to simply use the mastermethod, and to save the substitution method forrecurrences that actually need its full power.


13/20

Consider the following reccurence relation, which showsup fairly frequently for some types of algorithms:

T(1) = 1T(n) = 2T(n1) + c1By expanding this out a bit (using the"iteration method"), we can guess that this will be O(2n). Touse the substitution method to prove this bound, we now needto guess a closed-form upper bound based on this asymptoticbound. We will guess an upper bound ofk2nb, where b issome constant. We include the b in anticipation of having todeal with the constant c1 that appears in the recurrencerelation, and because it does no harm. In the process ofproving this bound by induction, we will generate a set of

constraints on k and b, and ifb turns out to be unnecessary, wewill be able to set it to whatever we want at the end.

Our property, then, is T(n) k2nb, for some twoconstants k and b. Note that this property logically impliesthat T(n) is O(2n), which can be verified with reference to thedefinition of O.


14/20

Base case: n = 1. T(1) = 1 k21 b = 2k b.

This is true as long as k (b + 1)/2.

Inductive case: We assume our property is

true for n 1. We now want to show that itis true for n.

T(n) = 2T(n1) + c1

2(k2n 1 b) + c1 (by IH)

= k2n 2b + c1

k2n b

This is true as long as b c1.


15/20

So we end up with two constraints that need to be satisfied forthis proof to work, and we can satisfy them simply byletting b = c1 and k = (b + 1)/2, which is always possible, as thedefinition of O allows us to choose any constant. Therefore, wehave proved that our property is true, and so T(n) is O(2n).

The biggest thing worth noting about this proof is theimportance of adding additional terms to the upper bound weassume. In almost all cases in which the recurrence hasconstants or lower-order terms, it will be necessary to haveadditional terms in the upper bound to "cancel out" theconstants or lower-order terms. Without the right additionalterms, the inductive case of the proof will get stuck in the

middle, or generate an impossible constraint; this is a signal togo back to your upper bound and determine what else needs tobe added to it that will allow the proof to proceed withoutcausing the bound to change in asymptotic terms.


16/20

The master methodis a cookbook method for solving recurrences.Although it cannot solve all recurrences, it is nevertheless very handy fordealing with many recurrences seen in practice. Suppose you have arecurrence of the form\

T(n) = aT(n/b) + f(n),

where a and b are arbitrary constants andfis some function ofn. Thisrecurrence would arise in the analysis of a recursive algorithm that forlarge inputs of size n breaks the input up into asubproblems each ofsize n/b, recursively solves the subproblems, then recombines theresults. The work to split the problem into subproblems and recombinethe results isf(n).

We can visualize this as a recurrence tree, where the nodes in the treehave a branching factor ofa. The top node has workf(n) associated withit, the next level has workf(n/b) associated with each ofa nodes, thenext level has workf(n/b2) associated with each ofa2 nodes, and so on.At the leaves are the base case corresponding to some 1 n < b. The treehas logbn levels, so the total number of leaves is a

logbn = nlogb

a.


17/20

The total time taken is just the sum of the time taken at eachlevel. The time taken at the i-th level is aif(n/bi), and the totaltime is the sum of this quantity as i ranges from 0 to logbn1,plus the time taken at the leaves, which is constant for each leaftimes the number of leaves, or O(nlogb

a).

T(n) = 0i


18/20

Case 3:f(n) is (nlogba+ ). In this casefgrows

faster than the number of leaves, which meansthat asymptotically the total amount of work isdominated by the work done at the root node.For the upper bound, we also need an extra

smoothness condition onfin this case, namelythat af(n/b) cf(n) for some constant c < 1 andlarge n. In this case T(n) is (f(n)).

As mentioned, the master method does notalways apply. For example, the second example

considered above, where the subproblem sizesare unequal, is not covered by the mastermethod.

Let's look at a few examples where the mastermethod does apply.


19/20

Example 1:

Say you have derived the recurrence relation T(n) =8T(n/2) + cn2, where c is some positive constant. Wesee that this has the appropriate form for applyingthe master method, and that a=8, b=2, and h(n)= cn2. cn2 is O(nlog28 ) = O(n3 ) for any 1, so thisfalls into case 1. Therefore, T(n) is (n3).

Example 2: Say you have derived the recurrence relation T(n)

= T(n/2) + cn, where c is some positive constant. Wesee that this has the appropriate form for applyingthe master method, and that a=1, b=2, and h(n) = cn.Then h(n) is (nlog2

1 + ) = (n) for any 1, so thisfalls into case 3. And ah(n/b) = cn/2 = h(n),therefore T(n) is (n).


20/20

Example 3:

Say you have derived the recurrence

relation T(n) = 8T(n/4) + cn3/2, where c is some

positive constant. We see that this has the

appropriate form for applying the mastermethod, and that a=8, b=4, and h(n)

= cn3/2. cn3/2 is (nlog48) = (n3/2), so this falls

into case 2. Therefore, T(n) is (n3/2log n).

Algorithm and Desing

Documents

Transcript of Algorithm and Desing