Recurrences

Oscar Civit oscar.civit@upf.eduDaniel Giribet daniel.giribet@upf.edu

What are recurrences and where do we find them?

Recurrences are functions defined on an integer domain that include themselves in their own definition.

Ex: T(n) = T(n/2) + n

They appear when trying to calculate the cost of recursive algorithms.

What are recurrences and where do we find them? (II)

MERGE-SORT sorting algorithm cost can be written as:

T(n) = (1) if n=12T(n/2) + (n) if n>1

Resolution

When analyzing the cost of an algorithm, we aim to find a closed formula that computes the asymptotic cost with the input size as the only variable. This asymptotic cost allows us to compare the efficiency of different algorithms.

There are 3 main methods to solve this kind of recurrences, which we will explain in detail later: Substitution method Iteration method Master method

Technicalities

Integer arguments

We usually ignore that the input size is an integer, omiting the use of floor/ceil operators in the formulas.

T(n) = T(floor(n/2)) + T(ceil(n/2)) + n = 2T(n/2) + n

Boundary conditions: T(n_0) = T_0

We usually omit boundary conditions when solving the recurrence, as most of the times they only affect the result by some constant which does not change asymptotic cost.

Substitution Method

Basic idea is making a reasonable guess of the solution to the recurrence and prove it by induction.

Ex: T(n) = 2T(n/2) + n

Guess:T(n) = O(n log n)

To be proved:T(n) 0

Substitution Method (II)

General case

We assume that it works for n/2, lets try for n:T(n)

Substitution Method (III)

Base case

Proving the base case is proving that the inequality holds for the boundary conditions of the recurrence. Due to the asymptotic notation, it is sufficient to prove it for some n_0.

In the example, for n_0 = 3:T(3) = 5T(3) = 2

How to become a Guess Star

Similar recurrences usually have similar solutions.If a recurrence is similar to another one they may have thesame asymptotic behavior.Ex: T_1(n) = 2T_1(n/2) + n = O(n log n)

T_2(n) = 2T_2(n/2 + K) + n = O(n log n) Sometimes a change of variable can be performed to

transform an unknown recurrence into a known one:T(n) = 2T(sqrt(n)) + log n

we change variable: m = log nT(2^m) = 2T(2^(m/2)) + mS(m) = T(2^m) = 2S(m/2) + m

which has a known result: S(m) = O(m log m)and changing back the variable:

T(n) = O(log n log( log n ))

How to become a Guess Star (II)

Prove loose upper and lower bounds and try to make them converge. Ex:We can start proving:

T(n) = (n)T(n) = O(n^2)

If we succeed we can raise the lower limit and lower the higher limit to finally prove:

T(n) = O(n log n)

T(n) = (n log n) And therefore:

T(n) = (n log n)

Iteration Method

1) Expand the recurrence formula iterating it until it can be expressed as a summation of terms dependent only on n and the initial conditions (boundary).

2) Use standard techniques for bounding summations to get the asymptotic cost.

Ex:T(n) = 3T(n/4) + nT(n) = n + 3T(n/4)

= n + 3( (n/4) + 3T(n/16) )= n + 3( (n/4) + 3( (n/16) + 3T(n/64) ) )= n + 3(n/4) + 9(n/16) + 27T(n/64)

Iteration Method (II)

T(n)

Master Method

Recipe for success, but only when the recurrence can be cast into the form:

T(n) = aT(n/b) + f(n)where

a >= 1b > 1f(n) is asymptotically positive

Master Method (II)

The Master Theorem is divided in 3 cases: a) If f(n) = O(n^log_b(a - e)) for some e > 0

then T(n) = (n^log_b a)

b) If f(n) = (n^log_b a) then T(n) = ( (n^log_b a) log n )

c) If f(n) = ((n^log_b(a + e)) for some e > 0 and if a f(n/b)

Example: the Hanoi Towers

Classic example of moving a tower of n blocks from one pleace to another under some restrictions.

Blocks cannot be placed on top of smaller ones.

There is a starting block, an auxiliary place and a final destination.

The Hanoi Towers: solution

A simple and intuitive solution to the problem can befound by induction

Case where n=1 is trivial, block is simply moved to proper place (base case) in one step

The Hanoi Towers: solution

To solve case where n>1, a smaller by one subset of blocks is taken (n-1)

The subset is smaller and we can suppose it solved by induction

Instead of placing it at destination, it is placed on intermediate area

Tasks left: Lowest block is yet unsolved and in original place Place n-1 subset on final destination

The Hanoi Towers: solution

Lowest block is trivially moved to proper place

By induction, smaller subset left is placed on the finaldestination (over just-moved lowest block)

The Hanoi Towers: solution

Base case solved

Method of moving from smaller subset of problem solved(by hyphotesis) to full completion

Subsets decrease to empty consistently

Solved! But what is the cost O() of the algorithm?

Cost can be described by a recurrence:T(n) = T(n-1)+1+T(n-1) = 2T(n-1)+1, with T(1)=1

The Hanoi Towers: cost

Recurrence is not in asymptotic form

The iteration method of solving can be attempted to get the asymptotic bound

Expanding (iterating) the sequence to depend only on n leads to:

T(n) = 2( 2( 2T(n-3)+1 )+1 )+1

The Hanoi Towers: cost

Further applying the accumulated multiply factors to the constants leads to:T(n) = 2^n-1 * T(1) + ... + 2^2 + 2 + 1

In this case, the series is well-known and easy to solve without resorting to more convoluted algebra, though by iterating the recurrence it is definitely far easier to come up with the solution

The solution in this case is trivial enough: T(n) = 2^n - 1