How are Generating Functions used in finding the closed ......Introduction _____ 4 Generating...

Candidate code: glt090 Date of submission: 10 May 2017

How are Generating Functions used in

finding the closed form of sequences

involving recurrence relations and in the

analysis of probability distributions?

Mathematics Extended Essay

Word count: 3865

Abstract word count: 209

“The fascination one can have with mathematics is similar to the fascination a boy can have with a girl.”

Hiroshi Yuki (2012)

ALL TEXTS BELONG TO OWNERS.TAKEN FROM WWW.INTERNALASSESSMENTS.WORDPRESS.COM


Contents

Important Equations and Identities ______________________________________________________ 2

Abstract ___________________________________________________________________________ 3

Introduction ________________________________________________________________________ 4

Generating Functions in Finding the Closed Form of Sequences Involving Recurrence Relations _____ 6

A simple sequence {2, 4, 8, 16 …}____________________________________________________ 6

The Fibonacci Sequence ____________________________________________________________ 7

Derivation of the sum of an infinite geometric series __________________________________10

The Catalan Numbers ______________________________________________________________16

Probability Generating Functions _______________________________________________________26

Uses of the PGF __________________________________________________________________27

The Binomial Distribution __________________________________________________________30

The Poisson Distribution ___________________________________________________________34

The Maclaurin Series__________________________________________________________35

Conclusion_________________________________________________________________________39

Works Cited________________________________________________________________________40

1



Equations and Identities used in this extended essay

Generating function for finding the closed form of a sequence:

(x) x x x x x .. x ..G = ∑∞

n=0un

n = u00 + u1

1 + u22 + u3

3 + . + unn + .

Probability Generating Functions:

(t) P (X )Gx = ∑∞

x=0tx = x

Sum of an infinite power series:

..1 + x + x2 + . = 11−x For and as x| | < 1 n → ∞

Maclaurin series:

(x) f (0)xf = ∑∞

n=0

1n!

n n

2



Abstract

The research question of this Extended Essay is “How are Generating Functions used in finding the

closed form of sequences involving recurrence relations and in the analysis of probability

distributions?” Generating functions are incredibly useful as they represent an infinite series of numbers

as a single function (Graham, Knuth and Patashnik, 1994). They are therefore a powerful tool in analysing

probability distributions and finding the closed form of a sequence. In investigating this research question,

I began by applying generating functions to finding the closed form of the Fibonacci Sequence and the

Catalan Numbers. The techniques I used in finding these closed forms helped me to approach finding the

probability generating functions of four probability distributions: The Binomial Distribution and The

Poisson Distribution. The investigation included approaches using the Maclaurin Series and the Binomial

Theorem.

The conclusion I came to was that the generating functions for any sequence or probability distribution is

highly useful, allowing one to analyse the sequence and probability distribution much more quickly and

efficiently. The research showed how generating functions analyse probability distributions, sequences

involving recurrence relations, and its applications in real life. This essay concludes with a summary of

findings and areas of exploration on Generating Functions.

(Word count: 209)

3



Introduction

“The most powerful way to deal with sequences of numbers, as far as anybody knows, is to manipulate

infinite series that generate those sequences.” (Graham, Knuth, and Patashnik, 1994, p320)

Generating functions are one of the most powerful ways of analysing a series of numbers, by expressing

the terms of a sequence as coefficients of a series expansion, and finding the closed form of that series

expansion. The concept of a generating function was introduced by Abraham de Moivre in 1730 (Knuth,

1973), and has been used since then to analyse sequences, to solve problems in combinatorics, computer

programming, and in analysing probability distributions.

Generating functions are functions which expresses an infinite sequence as coefficients of a series

expansion. Here, un is the nth term of the sequence as coefficients of x, an indeterminate which cannot be

solved for. It is simply a symbol or placeholder. A generating function is not a true “function” which

maps a domain and range but is more of a series which obeys the laws of a function, such as having

derivatives and integrals and can be manipulated by the rules following a regular function. Below is the

general form of a generating function:

(x) xG = ∑∞

n=0un

n

Where x is an indeterminate or variable, and n is a positive whole number.

By calculating the function which produces this infinite series, one can generalise the sequence and find

its closed form. This may involve calculus, algebra, and the use of Maclaurin Series, falling factorials or

other methods in order to find the closed form of the function. In this essay, a closed form is defined as an

expression which generates the term by simply knowing the nth term.un

4



The subject of sequences and generating functions was the first one to interest me - especially because of

the Fibonacci Sequence, which appears in nature, art, and poetry. It is a recurrence relation, making it

difficult to generalise, but generating functions can find the closed form of this sequence and others which

also are in the form of recurrence relations. The Catalan Numbers and the Fibonacci sequence have

important roles in combinatorics and finding the general form of these sequences are therefore important.

Statistical analysis is also another area in which Generating Functions play an important role, as they can

help facilitate the finding of the mean, variance, and probabilities of a probability distribution. As we shall

see, the product of probability generating functions of independent variables corresponds to their sum,

and this greatly increases the efficiency of computations relating to probability distributions.

This extended essay relates to the topics in the Math HL course of Statistics and Probability, Functions

and Equations, and was written before we began studying Statistics and Probability, which helped me

greatly when we began the topic.

The research question, “How are Generating Functions used in finding the closed form of sequences

involving recurrence relations and in the analysis of probability distributions?” is the focus of this

extended essay, I will be showing the use of generating functions in finding the closed form of a sequence

and its use in probability distributions in this extended essay.

5



Generating Functions in Finding the Closed Form of Sequences Involving

Recurrence Relations

A simple sequence: {2, 4, 8, 16 …}, or run = u1n−1

I used a simple sequence first to illustrate how generating functions work. In this case, the generating

function of the sequence shown above would be

(x) x x x 6x ...G = 2 + 4 2 + 8 3 + 1 4 (refer back to general form on page 2)

Normally, the general form of this sequence would be relatively easy to figure out by comparison.

However, for the sake of demonstrating the process of using a generating function, I will show how to

find the general form of the sequence using the generating function method.2, 4, 8, 16 ...}{

Since we know the first few terms of the sequence, we make an infinite sum using the sequence as the

coefficients of a power series.

(x) x x x 6x ..G = 2 + 4 2 + 8 3 + 1 4 + .

From this we can generalise (the nth term of the sequence) asun

(x)G x= ∑∞

n=02n n where n is a positive whole number

And since we know the generalisation of the generating function,

un = 2n

This gives us the general form of the sequence {2, 4, 8, 16...}.

6



Now that I have demonstrated the use of generating functions in finding the closed forms of simple series,

I will now move on to more complicated ones such as series containing recurrence relations. One of the

most famous examples of this is the Fibonacci sequence.

The Fibonacci Sequence

Arithmetic and geometric sequences, such as in the above example, are quite easy to find the general

formula as they have common ratios and differences which can be easily found using division or

subtraction. However, there are other sequences that exist such as the Fibonacci sequence, which is in the

form of a recurrence relation.

The Fibonacci sequence can be expressed as the sequence

{0, 1, 1, 2, 3, 5, 8, 13, 21, 34 …}

Where the nth term is the sum of the and term, such thatn )( − 1 th n )( − 2 th

u0 = 0

u1 = 1

, n > 2un = un−1 + un−2

The generation of the nth term as a function of the previous terms is what makes the Fibonacci sequence a

recurrence relation.

The general form of recurrence relations are generally harder to find as there is no common ratio or

common difference or general formula to express the sequence. Although there is a pattern, this is not

7



practical for larger numbers n, thus generating functions are used to find the general forms of sequences

which depend on recurrence relations.

The generating function of the Fibonacci sequence would be like so (where is power series with (x)G

coefficients being the terms of the Fibonacci sequence):

(x) x x x x x ..G = 0 0 + 1 1 + 1 2 + 2 3 + 3 4 + .

(x) x x ..G = x + x2 + 2 3 + 3 4 + .

Or, in a more general form,

(x) x x x x ..G = u00 + u1

1 + u22 + u3

3 + .

Generalized as

(x) .. x x x ..G = . + un−2n−2 + un−1

n−1 + unn + .

However, it’s nearly impossible to add the terms together in this form as the powers of x are different.

I noticed that the recurrence relation of the Fibonacci sequence allowed me to “equalize” the powers of x,

by multiplying by powers of x.(x)G

x xun−2n−2 · x2 = un−2

n

x xun−1n−1 · x1 = un−1

n

I multiplied G(x) by x0, x1, and x2 in order to obtain the following equations:

(x) x x x x ..G · x0 = u00 + u1

1 + u22 + u3

3 + . ………… 1

(x) u x x x x ..G · x1 = 01 + u1

2 + u23 + u3

4 + . ………… 2

8



(x) u x x x x ..G · x2 = 02 + u1

3 + u24 + u3

5 + . ………… 3

Subtracting 1 from 3 + 2:

Left-hand side = (x) (x) (x)G · x2 + G · x1 − G · x0

= (x)(x )G 2 + x − 1

Right-hand side = x xu x01 + u0

0 − u11

u )+ ( 0 + u1 − u2 · x2

u )+ ( 1 + u2 − u3 · x3

u )+ ( 2 + u3 − u4 · x4

..+ .

u )+ ( n−2 + un−1 − un · xn

..+ .

From the general form of this combined equation I saw that

u ) u u )]( n−2 + un−1 − un · xn = [ n−2 + un−1 − ( n−2 + un−1 · xn

u − )= ( n−2 + un−1 − un−2 un−1 · xn

= 0

Everything in this equation will cancel out except for the first three terms, leaving the equation

(x)(x ) x xG 2 + x − 1 = u x01 + u0

0 − u11

9



And since we know what u0 and u1, which are 0 and 1 respectively.

(x)(x ) −G 2 + x − 1 = x

(x)G = x1−x−x2

This is the generating function of the Fibonacci Sequence. This is the sum of the infinite series which has

coefficients of the Fibonacci Sequence.

However, this generating function needs to be converted into a sequence form in order for it to allow one

to generate the nth term of the sequence. Currently, we know

(x)G = x1−x−x2

I realised that in order to convert the Fibonacci Sequence into a generalised sequence, I would need to

make use of the sum of an infinite geometric series (Yuki, 2012).

Derivation of the sum of an infinite geometric series

Consider:

1)(1 )( − x = 1 − x1

→ 1 )(1 )( + x − x = 1 − x2

→ 1 )(1 )( + x + x2 − x = 1 − x3

→ 1 )(1 )( + x + x2 + x3 − x = 1 − x4

We can generalise this:

1 .. )(1 )( + x + x2 + . + xn − x = 1 − xn+1

10



Creating the equation

1 .. )( + x + x2 + . + xn = 1−x1−xn+1

And when , we can see thatx| | < 1

limx→∞

xn+1 = 0

Therefore for and as ,x| | < 1 n → ∞

..1 + x + x2 + . = 11−x

For the function , we could also express this as (x) ..f = 1 + x + x2 + x3 + x4 + .

where the coefficients of x form the sequence {1, 1, 1, 1, 1,(x) x x x x x ..f = 1 0 + 1 + 1 2 + 1 3 + 1 4 + .

1...}. The general form of a generating function is as follows:(x)G

(x)G x x x x ..= u00 + u1

1 + u22 + u3

3 + .

x= ∑∞

n=0un

n

Now, back to the Fibonacci Sequence. This helped me in expressing the function G(x) as an infinite

series. I began by factorizing the bottom of the fraction in G(x), taking into account the fact that x is an

indeterminate. Consider:

x1−x−x2 = R

1−rx + S1−sx

= (1−rx)(1−sx)R(1−sx)+S(1−rx)

11



= 1−sx−rx+rsx2R−Rsx+S−Srx

= 1−(s+r)x+rsx2(R+S)−(Rs+Sr)x

By using comparison, we can see that the coefficients come out to

1. R + S = 0

2. s r − R + S = 1

3. s + r = 1

4. s − r = 1

We can express R and S in terms of r and s .

R = − S

s r − S + S = − 1 (by substituting equation 1. Into equation 2.)

(r ) − S − s = 1

S = − 1r−s

Similarly, we can find that

R = 1r−s

And, using this, we can therefore form the expression

12



(x) G = x1−x−x2

= R1−rx + S

1−sx

= 1r−s · 1

1−rx − 1r−s · 1

1−sx

( )= 1r−s

11−rx − 1

1−sx

We can see here how and correspond to the generating function of . We replace 11−rx 1

1−sx 11−x 1

1−rx

and with their infinite sums, referring back to the infinite sum on page 11.11−sx

(x) ((1 x x x ..) 1 x x x ..)) G = 1r−s + r + r2 2 + r3 3 + . − ( + s + s2 2 + s3 3 + .

((r )x r )x r )x ..) = 1r−s − s + ( 2 − s2 2 + ( 3 − s3 3 + .

x x x ..= r−sr−s + r−s

r −s2 2 2 + r−sr −s3 3 3 + .

Which can be expressed in the general form,

(x) xG = ∑∞

n=0r−s

r −sn n n

And since we know that

(x) xG = ∑∞

n=0un

n

13



We can therefore deduce that

un = r−sr −sn n

Where un is the nth term of the infinite sum.

Now, we also know that

s + r = 1

And s − r = 1 from page 12

And since r and s are the roots of the equation

s )x s x2 − ( + r + r = 0

x2 − x − 1 = 0

We can use the quadratic formula to solve for the roots of the equation.

x = 2a−b±√b −4ac2

→ x = 21±√1+4

→ x = 21±√5

Assuming that , r > s

14



r = 21+√5

and s = 21−√5

r − s = 21+√5 − 2

1−√5

= 21−1+2√5

= √5

This shows that

(( ) ) )un = 1√5 2

1+√5 n − ( 21−√5 n

Which is the general form of the Fibonacci sequence.

We can test this out:

(( ) ) ) (1 )u0 = 1√5 2

1+√5 0 − ( 21−√5 0 = 1

√5 − 1 = 0

(( ) ) ) ( )u1 = 1√5 2

1+√5 1 − ( 21−√5 1 = 1

√5√5 = 1

From this process, we can see that using a generating function to find the general form of a sequence

follows these steps:

1. Express the generating function of the sequence in the form of an infinite power series, using the

formula (x) xG = ∑∞

n=0un

n

2. Express the infinite power series as an infinite sum

15



3. Find the general form of in the form in which is an expression by itself(x)G x∑∞

n=0un

n un

4. Extract from step 3, finding the closed form of the sequenceun

Now, I will demonstrate this method of using generating functions to find the general form, of a un

sequence containing recurrence relations using another sequence: the Catalan numbers.

The Catalan Numbers

The Catalan numbers are another famous mathematical sequence which had stumped mathematicians

since their publication in the work of Eugene Charles Catalan as a solution to a combinatorics problem,

similar in the way that the Fibonacci numbers had been “discovered” multiple times but made famous by

one mathematician. The Catalan numbers appear in many situations, one of which is a round table around

which 2n number of people sit. The Catalan numbers (un being the nth term) are then the number of ways

those 2n people can shake hands without their arms crossing. Where un is the nth term of the sequence.

The Catalan numbers consist of the following sequence (from n = 1):

Table 1: The Catalan Numbers

n 1 2 3 4 5 6 7 8

un 1 2 5 14 42 132 429 1430

Another combinatorics problem which could be used to find a solution to the Catalan numbers would be

how many different ways to rearrange brackets for n+1 numbers, where A is any arbitrary number. We

are not actually adding any numbers but rather just using them as placeholders so we can rearrange the

number of brackets in order to illustrate the Catalan numbers.

16



u0 = 1.

For n=1, way to arrange the bracketsA ) A + A = ( + A = 1

For n=2, A + A + A (A ) )= ( + A + A → u1 · u0

A A ))= ( + ( + A → uo · u1

ways to arrange the brackets= 2

For n=3, A + A + A + A ((A ) ) )= ( + A + A + A

(A A )) )= ( + ( + A + A

(A ) A ))= ( + A + ( + A includes arrangement of two n = 1

A (A ) )= ( + ( + A + A

A A A ))= ( + ( + ( + A

ways to arrange the brackets = 5

There’s a pattern emerging in these examples - one can see that there is a grouping of in :n = 2 n = 3

Therefore we can see that affects the value of . un will be a sum of the previous sums, or that,un−1 un

u2 = u0 · u1 + u1 · u0

u3 = u0 · u2 + u1 · u1 + u2 · u0

17



From this, we can generalise that

,u0 = 1

u u u .. u .. uun+1 = u0 n−0 + u1 n−1 + u2 n−2 + . + uk n−k + . + un−0 0

Or,

,u0 = 1

uun+1 = ∑n

k=0uk n−k

This is the recurrence relation of the Catalan Numbers.

Now for the generating function of the Catalan numbers:

(x) x x x x .. x ..G = u00 + u1

1 + u22 + u3

3 + . + unn + .

→ (x) x x x .. x ..G = u0 + u1 + u22 + u3

3 + . + unn + .

→ (x) xG = ∑∞

n=0un

n

In order to find the closed form of , I tried a few methods such as differentiation and subtraction (the (x)G

technique I used for the Fibonacci sequence) but eventually found that squaring was the most (x)G

effective way to find the closed form of the generating function.

(x) u u u u )x u u u u )x .. u )x ..G 2 = u0 0 + ( 0 1 + u1 0 + ( 0 2 + u1 1 + u2 02 + . + ( ∑

n

k=0uk n−k

n + .

Showing that the coefficients of are .xn u∑n

k=0uk n−k

18



(x) ( u )xG 2 = ∑∞

n=0∑n

k=0uk n−k

n

And since we know that uun+1 = ∑n

k=0uk n−k

(x) xG 2 = ∑∞

n=0un+1

n

In a similar fashion to the Fibonacci sequence, we can manipulate this equation by multiplying it by x.

Doing so will change the power of x in the current equation and allow us to find the closed form of the

Catalan numbers.

(x) xx · G 2 = x ∑∞

n=0un+1

n

→ (x) xx · G 2 = ∑∞

n=0un+1

n+1

→ (x) xx · G 2 = ∑∞

n+1=1un+1

n+1

I added 1 to both sides of the lower limit so n + 1 would be achieved, equal to the notation for u and x.

This can be further generalised to

(x) xx · G 2 = ∑∞

n=1un

n

(x) xx · G 2 = ∑∞

n=0un

n − u0

19



Knowing that ,(x) xG = ∑∞

n=0un

n

(x) (x)x · G 2 = G − u0

And since we know that u0 = 1,

G(x) (x)x 2 − G + 1 = 0

Using the quadratic formula,

(x)G = 2x1±√1−4x

However the complicates things. I realised that I couldn’t have two closed forms±

x (x) 2 · G = 1 + √1 x− 4 or x (x) 2 · G = 1 − √1 x− 4

For x=0,

(0) (0) 2 · G = 1 + √1 (0)− 4 or (0) (0) 2 · G = 1 − √1 (0)− 4

= 0 / 1 + √1 0 = 0

Therefore, we know that the correct closed form of G(x) is

(x)G = 2x1−√1−4x (*)

In order to find the closed form of the generating function, we’ll attempt to find the closed form of the

power series which corresponds to the generating function G(x), as we did for the Fibonacci sequence.

20



Because G(x) is one power series ( ), and the expansion of can also be expressed as a x∑∞

k=0uk

k √1 x− 4

power series ( ) , a generating function I will call so that will not be confused with it. x∑∞

k=0P k

k (x)P (x)G

The coefficients of the generating function will be to avoid confusion with the Catalan Numbers. (x)P P k

This would mean that the equation for G(x) would now be:

x x x2 · ∑∞

k=0uk

k = 1 − ∑∞

k=0P k

k From (*)

→ x P x )∑∞

k=02 · uk−1

k+1 = 1 − ( 0 + ∑∞

k=1P k

k

→ x x∑∞

k=12 · uk−1

k = 1 − P 0 − ∑∞

k=1P k

k

→ x x∑∞

k=12 · uk−1

k + ∑∞

k=1P k

k = 1 − P 0

→ (2 )x∑∞

k=1· uk−1 + P k

k = 1 − P 0

By comparing the coefficients of the terms, I found that

0 = 1 − P 0

u2 0 + P 1 = 0

u2 1 + P 2 = 0

u2 2 + P 3 = 0

…

21



u2 n + P n+1 = 0

We can generalise this as

P 0 = 1

−un = 2P n+1

Therefore if we find the general form of P n+1 we would be able to find the closed form of the generating

function . We know that(x)G

(x) P = √1 x− 4

(x) x x x x ..P = P 00 + P 1

1 + P 22 + P 3

3 + .

Applying the rules we use for regular function to P(x), differentiating we get

(x) P x P x P x ...P ′ = P 1 + 2 2 + 3 32 + 4 4

3

(x) P P x P xP ′′ = 1 · 2 2 + 2 · 3 3 + 3 · 4 43

Thus (0) PP ′ = 1 1

(0) PP ′′ = 2 2

(0) PP ′′′ = 3 · 2 · 1 3

Or generalised as,

(0) !PP (n) = n n

P n = n!P (0)(n)

22



After I arrived at this, I realised that I needed to link the equation to the identity of by (x) P = √1 x− 4 P n

finding the nth derivative of .(x)P

(x) P = 1 x) ( − 4 21

→ (x) − (1 x) P ′ = 2 − 4 − 21

→ (x) − (1 x) P ′′ = 2 · 2 − 4 − 23

→ (x) − (1 x) P (3) = 2 · 4 · 3 − 4 − 25

→ (x) − (1 x) P (4) = 2 · 6 · 5 · 4 − 4 − 27

→ (x) − (1 x) P (5) = 2 · 8 · 7 · 6 · 5 − 4 − 27

Here I realised that I needed some sort of annotation for the factorial which ended on a number which was

not one, for example without using fractions as this would complicate calculations further on if 8 · 7 · 6 · 5

I attempted to use fractions and factorials. After researching, I found that the most effective method was

the falling factorial, which is a way of expressing the factorials of a number n number of times. For

example:

x) x)(x )(x )(x )(x )...(3)(2)(1) !( x = ( − 1 − 2 − 3 − 4 = x (x goes through the factorial x times)

x) x)(x )( 2 = ( − 1 (n goes through the factorial twice)

x) x)(x )(x )(x )(x )( 5 = ( − 1 − 2 − 3 − 4 (n goes through the factorial five times)

23



Generalised as

x) (x )( n = ∏n−1

k=0− k

Where is similar to the sign but multiplies instead of adding .∏

∑

I used this notation in writing the generalisation of the identity of , which is below.(x)P (n)

(x) − 2n ) (1 x)P (n) = 2 · ( − 2 n−1 − 4 − 2

2n−1

→ (x) − (2n) (1 x) P (n+1) = 2 n − 4 − 22n+1

→ (0) − (2n)P (n+1) = 2 n

And from the equation, :P n = n!P (0)(n)

P n+1 = (n+1)!P (0)(n+1)

= −2(2n)n(n+1)n+1

We can substitute this into the equation −un = 2P n+1

∴ un −= 2P n+1 (from page 21)

= (2n)n(n+1)n+1

(substituting what we know of Pn+1)

= (2n)n(n+1)(n)n

(separating the falling factorials)

24



From here, we know that

2n) n(2n )(2n )...(n )( n = 2 − 1 − 2 + 1

→ 2n) (n) n(2n )(2n )...(n )(n)(n )(n )...(3)(2)(1)( n n = 2 − 1 − 2 + 1 − 1 − 2

→ 2n)( n = n!(2n)!

Then we can put that back into the previous equation where we separated the falling factorials:

( )= 1n+1 n!n!

(2n)!

( )= 1n+1

(2n)!n!(2n−n)!

( )= 1n+1 n

2n

This is the general form of the nth term of the Catalan numbers.

After exploring the use of generating functions in finding the general form of a sequence defined by

recurrence relations, I decided to look at a more practical approach - the applications of generating

functions in statistics.

25



Probability Generating Functions

Generating functions are used in probability and statistics to predict the probability of an event occurring

as the coefficient of an indeterminate, t (it does not have a value but can be treated like a variable in a

regular function), which has the power x, such that

For a probability distribution X,

(t) (t ) P (X )GX = E x = ∑∞

x=0tx = x

Where is the probability generating function (PGF) of a probability distribution X (the number of (t)GX

successes or occurrences as defined in each different distribution), where P(X = x) is the probability

distribution.

The PGF allows one to find a lot of information about a probability distribution much more simply than

other methods. It has the following uses (Chapter 4: Generating Functions, n.d.):

1. Finding for any value of x as a coefficient of tx(X ) P = x

2. Finding the mean of a probability distribution

3. Finding the variance of a probability distribution

4. Finding the sum of independent, discrete random variables

In this section, I will explore two different distributions and the derivations of their PGFs - the Binomial

Distribution and the Poisson Distribution.

26



Uses of the PGF

Firstly, the derivatives of the PGF can be used to generate the probabilities of the probability distribution

the PGF corresponds to (6 - Probability Generating Functions, n.d.).

(t) P (X ) (0) P (1) P (2) P (3) ..GX = ∑∞

x=0tx = x = P + t + t2 + t3 + .

→ (t) P (1) tP (2) t P (3) t P (4) ..G′X = 1 + 2 + 3 2 + 4 3 + .

→ (t) 2 )P (2) 3 )tP (3) 4 )t P (4) ..G′′X = ( × 1 + ( × 2 + ( × 3 2 + .

→ (t) 3 )P (3) 4 )tP (4) ..G′′′X = ( × 2 × 1 + ( × 3 × 2 + .

I used this formula to generalise the probabilities of any probability distribution by manipulating the

derivatives of the PGF:

(X ) (0)P = 0 = GX

(X ) (0)P = 1 = G′X

(X ) G (0)P = 2 = 12×1 ′′X

(X ) G (0)P = 3 = 13×2×1 ′′′X

(X ) G (0)P = 4 = 14×3×2×1

(4)X

And can be generalised to:

(X ) G (0)P = n = 1n!

(n)X

The mean and variance can also be found from the PGF of any probability distribution.

27



For the derivation of the mean, consider the PGF of the probability distribution X:

(t) P (X )GX = ∑∞

x=0tx = x

(t)G′X P (X ) tP (X ) t P (X ) t P (X ) ..= 1 = 1 + 2 = 2 + 3 2 = 3 + 4 3 = 4 + . Where (t) G (t)G′x = ddx x

There is a pattern in this - the series looks like the expectation of the probability distribution; however, the

only term that is out of place is t which can be replaced by 1 in order to find the expectation of the

probability distribution.

→ (1)G′X P (X ) P (X ) P (X ) P (X ) ..= 1 = 1 + 2 = 2 + 3 = 3 + 4 = 4 + .

P (X )= ∑∞

x=1x = x

(X)= E

This shows us that the mean of the probability distribution can be found from the PGF of a probability

distribution, rather than having to sum the probability distribution in the usual way:

Recall: (X) P (X )E = ∑

x = x

And for large probability distributions one would have to sum all the values of x and in order to (X )P = x

find the value of the expectation and therefore the PGF makes it much more efficient to find the

expectation of the probability distribution.

For the variance, I differentiated the PGF twice.

28



(t) P (X )GX = ∑∞

x=0tx = x

→ (t) t P (X )G′X = ∑∞

x=0x x−1 = x

→ (t)G′′X (x )t P (X )= ∑∞

x=0x − 1 x−2 = x

→ (1)G′′X (x )P (X ) (X(X )) (X ) (X ) (X)= ∑∞

x=0x − 1 = x = E − 1 = E 2 − X = E 2 − E

Where t = 1

For variance,

ar(X) V (X ) E(X)]= E 2 − [ 2

(X ) G (1)]= E 2 − [ ′X2

It seemed as if we were stuck here. However, we know that , so we can (1) (X ) (X)G′′X = E 2 − E

manipulate this equation:

ar(X) V (X ) (X) (X) G (1)]= E 2 − E + E − [ ′X2

(1) (X) G (1)]= G′′X + E − [ ′X2

(1) (1) G (1)]= G′′X + G′X − [ ′X2

Finally, the PGFs of two independent random distributions can be used to find the sum of those

independent variables. For example, if , and Z and Y are independent probability distributions, X = Z + Y

we can label their generating functions as and respectively. If we want to find the sum of the (t)GZ (t)GY

29



two distributions, or the generating function of the two distributions, we could do so using the generating

functions of Z and Y .

(t) (t ) (t ) (t ) (t ) (t ) (t) (t)GX = E x = E z+y = E z × ty = E z × E y = GZ × GY

Hence, we can see that a PGF is immensely useful in finding everything that there is to know about a

probability distribution.

In the following section, I will consider well-known probability distributions, their PGFs, and how

effectively information about the probability distributions can be found using the PGFs.

Binomial Distribution

The Binomial Distribution is a probability distribution limited by the following rules:

1. The trials are independent and are random variables

2. There are only two possible outcomes - “success” and “failure”

3. The probability of each trial is constant

The Binomial Theorem uses the Bernoulli Distribution, which is shown in Table 2 , but is repeated n

number of times.

Table 2: The Bernoulli Distribution

x 0 1

P(X = x) 1 - p p

30



Where p is defined as the probability of the success occurring and 1-p is the probability of the failure, X is

the random variable denoting the number of successes, and P(X = x) denotes the probability that X can

assume the value of x which is success or failure.

The Binomial Distribution occurs when an experiment is repeated n number of times, the probability of

success is p, and where X is the number of successes in these trials. The probability density function

(PDF) of the Binomial Distribution is:

(n, ) (X ) p (1 )X ~ B p ⇒ P = x = ( )xn x − p n−x , , ...nx = 0 1 2

The PGF of this Binomial Distribution or would therefore be(t)GX

(t)GX P (X ) P (X ) P (X ) P (X ) ..= t0 = 0 + t1 = 1 + t2 = 2 + t3 = 3 + .

( )p (1 ) ( )p (1 ) ( )p (1 ) ( )p (1 ) ..= t0 0n 0 − p n + t1 1

n 1 − p n−1 + t2 2n 2 − p n−2 + t3 3

n 3 − p n−3 + .

(tp) (1 ) (tp) (1 ) (tp) (1 ) (tp) (1 ) ..= ( )0n 0 − p n + ( )1

n 1 − p n−1 + ( )2n 2 − p n−2 + ( )3

n 3 − p n−3 + .

Let ,q = 1 − p

(t)GX (tp) q (tp) q (tp) q (tp) q ..= ( )0n 0 n + ( )1

n 1 n−1 + ( )2n 2 n−2 + ( )3

n 3 n−3 + .

Recall that (the Binomial Theorem)a ) a b( + b n = ∑n

r=0( )r

n n−r r

Therefore,

(t)GX q p)= ( + t n

1 p)= ( − p + t n

This is the PGF of the Binomial Distribution.

31



One of the uses of the PGF is that the expectation and variance of the PDF can be found very easily. The

expectation of the Binomial distribution can be found easily using the following method:

(X) (1) (t) n(q p) pE = GX ′ = GX ′ = p + 1 n−1 = n

Similarly, the variance of a distribution can be found using the PGF of that distribution. On page 29, the

variance of a distribution was defined as:

ar(X) (1) (1) G (1)]V = G′′X + G′X − [ ′X2

And can be used to find the variance of a distribution very quickly:

ar(X) V n(n )(q ) p np)= p2 − 1 + p n−2 + n − ( 2

p (n ) p np)= n 2 − 1 + n − ( 2

(n p p p)= p 2 − n + n − n2

p(1 )= n − p

pq= n

Next I asked myself the question: how can I work backwards from a PGF to find the original probability

density function? We can use this PGF in a similar way we used the generating function to find the nth

term of a sequence with the Fibonacci numbers, by re-writing the PGF in a form which is an infinite

series, allowing us to extract the probability density function.

(t)GX q p)= ( + t n

→ (t) n(q p)GX ′ = p + t n−1

32



→ (t) n(n )(q p)GX ′′ = p2 − 1 + t n−2

→ (t) n(n )(n )(q p)GX ′′′ = p3 − 1 − 2 + t n−3

Generalised as:

(t) (n )(n )(n )...(n )(q p)GX(y) = py − 1 − 2 − 3 − x + 1 + t n−y

→ (t) q p)GX(y) = py · n!

(n−y)! · ( + t n−y

→ (t) q p)GX(y) = py · n!y!

y!(n−y)! · ( + t n−y

→ (t) ! q p)GX(y) = py · y · n!

y!(n−y)! · ( + t n−y

→ (t) ! q p)GX(y) = py · y · ( )y

n · ( + t n−y

And since we know that (X ) (0)P = y = 1y! · G(y)

(X ) q)P = y = ( )yn · py · ( n−y

(X ) 1 )P = y = ( )yn · py · ( − p n−y

Which follows a Binomial Distribution. Using this method, we can prove that a generating function is of a

certain probability distribution or can find the probability density function of the distribution. One other

distribution which I looked at was the Poisson Distribution.

33



Poisson Distribution

The PDF of the Poisson Distribution is defined as follows:

o(λ) (X ) eX ~ P ⇒ P = x = x!λx −λ , 2, 3, 4, .... x = 1

Where

is the mean of the variable λ

x is the number of successes.

The Poisson Distribution is used to predict independent events over a certain period of time. It is the

number of x events that will happen, given that events happen on average over a certain period of time. λ

This distribution is used in the prediction of crime rates in a certain area, the number of deaths in war, the

number of hungry people entering a restaurant at a given period of time (Letkowski, 2012.) and many

other applications which have independent events happening over a certain period of time. The Poisson

Distribution can also be applied spatially - such as the number of errors on a page of writing, or the

number of defects on a certain product.

Finding the PGF of the Poisson Distribution is therefore a very valuable tool which can be used in

industry, criminology, as well as a multitude of other applications.

From the definition of the Poisson Distribution, we can see that the PGF will be

(t)GX P (X )= ∑∞

x=0tx = x

e= ∑∞

x=0tx x!

λx −λ

34



= e−λ ∑∞

x=0x!

(tλ)x

From this point it may seem almost impossible to find the PGF of the Poisson Distribution. However, we

must make use of Maclaurin’s series to simplify the result and to find the PGF of the series.

The Maclaurin series

The Maclaurin series is a Taylor Series centred on zero. The Maclaurin Series makes use of an

infinite series of polynomials to represent a function. This series is derived from the derivatives of

the original function (Khan, 2011). A function could therefore be expressed in a series of (x)f

numbers.

(x) x x x x ..f = a0 + a1 + a2

2 + a33 + a4

4 + .

→ (x) x x x ..f ′ = a1 + 2 · a21 + 3 · a3

2 + 4 · a43 + .

→ (x) x x x ..f ′′ = 2 · a2 + 3 · 2 · a31 + 4 · 3 · a4

2 + 5 · 4 · a53 + .

→ (x) x x ..f ′′′ = 3 · 2 · 1 · a3 + 4 · 3 · 2 · a4 + 5 · 4 · 3 · a52 + .

Substituting 0 for x, we can see that

(0)f = a0

→ (0)f ′ = a1

→ (0) af ′′ = 2 2 → a2 = 2f (0)′′

→ (0) !·f ′′′ = 3 a3 → a3 = 3!f (0)′′′

35



→ (0) !f (n) = n · an → an = n!f (0)(n)

So we can generalise that

(x) (0) (0)x f (0)x f (0)x f (0)x ... f (0)x ..p = f + f ′ + 21 ′′ 2 + 1

3×2 ′′′ 3 + 14×3×2

(4) 4 + . + 1n!

n n + .

While researching, I found that the Maclaurin series for was important to finding the PGF for the ex

Poisson Distribution. This led me to finding the Maclaurin Series for . has the handy quality, f (x) = ex ex

such that , leading to the Maclaurin Series for being very simple to find.e ex = ddx

x ex

ex x e x e x e x .. e x .. = e0 + e0 + 12!

0 2 + 13!

0 3 + 14!

0 4 + . + 1n!

0 n + .

.. ..= 1 + x + 2!x2 + 3!

x3 + 4!x4 + . + n!

xn + .

= ∑∞

n=0n!xn

After I found the Maclaurin Series of , I realised why it was important to finding the PGF of the ex

Poisson Distribution. What had seemed like a series that couldn’t be simplified into a closed form was

now made easier when I compared the Maclaurin Series of with the PGF of the Poisson Distribution. ex

(t)GX = e−λ ∑∞

x=0x!

(tλ)x

Since , therefore and we can substitute this value into the equation to get:ex = ∑∞

n=0n!xn etλ = ∑

∞

x=0x!

(tλ)x

→ (t) eGX = e−λ tλ

→ (t)GX = eλ(t−1)

36



Similarly to the Binomial distribution, how can we prove that this generating function represents the

Poisson Distribution or “work backwards” to find the original probability density function? This can be

achieved in the following steps:

(t) GX = eλ(t−1)

(t) GX ′ = λ · eλ(t−1)

(t) ·GX ′′ = λ2 eλ(t−1)

(t)GX ′′′ = λ3 · eλ(t−1)

Generalised as:

(t)GX(n) = λn · eλ(t−1)

Which can be substituted into the equation (X ) (0)P = n = 1n! · G(n)

(X )P = n = n!λ ·en −λ

Which is a Poisson Distribution.

Again, this shows us how to find the PDF of a distribution from its PGF.

Other than this, One can use the PGF to find the expectation of the distribution much more efficiently

than the “usual” method. In the introductory section of PGFs, I showed how one can use the PGF to find

the expectation. The expectation of a probability distribution is:

(X) P (X )E = ∑∞

x=0x = x or (X) (1)E = GX ′

With the first equation, for the Poisson Distribution, we would have to find the expectation through

solving this:

37



Solving for the expectation of the Poisson distribution is very simple if this equation is used:

(X) (t) E = GX ′ = λ · eλ(1−1) = λ

Using the PGF of the Poisson Distribution is very efficient as one simply has to differentiate it once and

then set x as 0.

Similarly, the variance of of the Poisson Distribution can be found using the following method:

ar(X) (1) (1) G (1)]V = G′′X + G′X − [ ′X2

·= λ2 eλ(1−1) + λ · eλ(1−1) − λ2

= λ

This shows clearly the efficiency of using the PGF in the analysis of probability distributions.

38



Conclusion

Through exploring this topic in the Extended Essay, I found that generating functions can be used to find

the closed form of sequences involving recurrence relations. Two of these sequences which I found the

closed form of were the Fibonacci Sequence and the Catalan Numbers. This shows the powerful

computational potential of generating functions, and how its properties can be used to apply methods of

manipulating functions to a sequence.

I also explored the use of PGFs in the analysis of probability distributions. Using the PGF of a

distribution, I found the individual probabilities as well as expectation and variance. The PGF can also be

used to find combinations of independent distributions using their individual PGFs. The PGF of a

distribution greatly reduces computational time for finding information about the probability distribution.

By using the various methods applied to regular functions, one can manipulate a probability distribution

to find information about it much more efficiently than if one did not use a PGF.

The scope of applications of generating functions is huge, ranging from its use in the sieve method in

combinatorics, the “Snake Oil” method for evaluating combinatorial sums, and many other combinatorial

problems. Computer algorithms also use generating functions and they can be used to prove combinatorial

identities. Sequences of functions can also be examined by generating functions, and the Legendre

polynomial generating function is important in the area of electrodynamics and vectors (Sahanggamu,

2006). These, especially examining the use of generating functions in combinatorics, could be topics for

further investigation.

39



Works Cited

6 - Probability Generating Functions. (n.d.). 1st ed. [ebook] Cambridge: University of Cambridge.

Available at: https://www.cl.cam.ac.uk/teaching/0708/Probabilty/prob06.pdf [Accessed 16 Dec.

2016].

Chapter 4: Generating Functions. (n.d.). 1st ed. [ebook] The University of Auckland. Available at:

https://www.stat.auckland.ac.nz/~fewster/325/notes/ch4.pdf [Accessed 5 Feb. 2017].

Graham, R., Knuth, D. and Patashnik, O. (1994). Concrete mathematics. 2nd ed. Reading:

Addison-Wesley, p.320.

Khan, S. (2011). Taylor & Maclaurin polynomials intro (part 1). [video] Available at:

https://www.khanacademy.org/math/calculus-home/series-calc/taylor-series-calc/v/maclauren-and-ta

ylor-series-intuition [Accessed 28 Dec. 2016].

Knuth, D. (1973). The art of computer programming. 1st ed. Reading: Addison-Wesley.

Letkowski, J. (2012). Applications of the Poisson probability distribution. 1st ed. [ebook] Springfield:

Western New England University. Available at:

http://www.aabri.com/SA12Manuscripts/SA12083.pdf [Accessed 26 Dec. 2016].

Sahanggamu, A. (2006). Generating Functions and Their Applications. Undergraduate. MIT Mathematics

Department.

Yūki, H. and Gonzalez, T. (2012). Math Girls . 2nd ed. Austin: Bento Books, Inc.

40


How are Generating Functions used in finding the closed ......Introduction _____ 4 Generating...

Documents

Transcript of How are Generating Functions used in finding the closed ......Introduction _____ 4 Generating...