R.Kass/Sp07 P416 Lecture 2 1 Binomial and Poisson Probability Distributions Binomial Probability...

R.Kass/Sp07 P416 Lecture 2 1

Binomial and Poisson Probability DistributionsBinomial Probability Distribution Consider a situation where there are only two possible outcomes (a “Bernoulli trial”)

Example: flipping a coin

The outcome is either a head or tail rolling a dice

For example, 6 or not 6 (i.e. 1, 2, 3, 4, 5)Label the possible outcomes by the variable kWe want tofind the probability P(k) for event k to occur

Since k can take on only 2 values we define those values as:k = 0 or k = 1

Let the probability for outcome “k” to occur be: P(k = 0) = q (remember 0 ≤ q ≤ 1)

something must happen so P(k = 0) + P(k = 1) = 1 (mutually exclusive events) P(k = 1) = p = 1 - q

We can write the probability distribution P(k) as:P(k) = pkq1-k (Bernoulli distribution)

coin toss: define probability for a head as P(1) P(k=1) = 0.5 and P(0=tail) = 0.5 too!

dice rolling: define probability for a six to be rolled from a six sided dice as P(k=1) P(k=1) = 1/6 and P(k=0=not a six) = 5/6.

James Bernoulli (Jacob I) born in Basel, SwitzerlandDec. 27, 1654-Aug. 16, 1705He is one 8 mathematicians in the Bernoulli family. (from Wikipedia)


What is the mean () of P(k)?

What is the Variance (2) of P(k)?

Let’s do something more complicated: Suppose we have N trials (e.g. we flip a coin N times) what is the probability to get m successes (= heads)? Consider tossing a coin twice. The possible outcomes are:

no heads: P(m = 0) = q2

one head: P(m = 1) = qp + pq (toss 1 is a tail, toss 2 is a head or toss 1 is head, toss 2 is a tail) = 2pq

two heads: P(m = 2) = p2

Note: P(m=0)+P(m=1)+P(m=2)=q2+ qp + pq +p2= (p+q)2 = 1 (as it should!) We want the probability distribution P(m, N, p) where:

m = number of success (e.g. number of heads in a coin toss)N = number of trials (e.g. number of coin tosses)p = probability for a success (e.g. 0.5 for a head)

kP(k)

k0

1

P(k)k0

1

0q1pq p

p

pqppppPPkP

kPk

k

k

)1()1(1)0(0)(

)(22222

1

0

1

0

2

2

we don't care which of the tosses is a head so there are two outcomes that give one head

Mean and Variance of a discrete distribution

(remember p+q=1)


If we look at the three choices for the coin flip example, each term is of the form:CmpmqN-m m = 0, 1, 2, N = 2 for our example, q = 1 - p always!

coefficient Cm takes into account the number of ways an outcome can occur without regard to order. for m = 0 or 2 there is only one way for the outcome (both tosses give heads or tails): C0 = C2 = 1

for m = 1 (one head, two tosses) there are two ways that this can occur: C1 = 2. Binomial coefficients: number of ways of taking N things m at time

0! = 1! = 1, 2! = 1·2 = 2, 3! = 1·2·3 = 6, m! = 1·2·3···m Order of occurrence is not important

e.g. 2 tosses, one head case (m = 1)we don't care if toss 1 produced the head or if toss 2 produced the head

Unordered groups such as our example are called combinationsOrdered arrangements are called permutations For N distinguishable objects, if we want to group them m at a time, the number of permutations:

example: If we tossed a coin twice (N = 2), there are two ways for getting one head (m = 1) example: Suppose we have 3 balls, one white, one red, and one blue.

Number of possible pairs we could have, keeping track of order is 6 (rw, wr, rb, br, wb, bw):

If order is not important (rw = wr), then the binomial formula gives

)!(!

!, mNm

NC N

mmN

PN,m N!

(N m)!

P3,2 3!

(3 2)!6

C3,2 3!

2!(3 2)!3 number of “two color” combinations


Binomial distribution: the probability of m success out of N trials:

p is probability of a success and q = 1 - p is probability of a failure

To show that the binomial distribution is properly normalized, use Binomial Theorem:

binomial distribution is properly normalized

Expectation Value = np = 50 * 1/3 = 16.667...

0.00

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0 5 10 15 20 25 30k

P(k

, 50,

1/3

)

Expectation Value = np = 7 * 1/3 = 2.333...

0.00

0.10

0.20

0.30

0.40

0 2 4 6 8 10k

P(k

, 7, 1

/3)

1)(),,(

)(

00

0

NmNmN

m

Nm

N

m

llkk

l

kl

k

qpqppNmP

baba

mNmmNmNm

mNmmN qp

mNm

NqpqpCpNmP

)!(!

!),,( ,


Mean of binomial distribution:

A cute way of evaluating the above sum is to take the derivative:

Variance of binomial distribution (obtained using similar trick):

Np

ppNp

ppmppppNqpmp

pmNpqpm

qpp

N

m

mNmNm

N

m

mNmNm

N

m

mNmNm

N

m

mNmNm

N

m

mNmNm

N

m

mNmNm

111

0

1

0

1

0

1

0

1

0

1

0

)1(1)1(

)1()1()1()1(

0)1)((

0

N

m

mNmNm

N

mN

m

N

m qpmpNmmPpNmP

pNmmP

00

0

0 ),,(),,(

),,(

2 (m )2 P(m, N, p)

m0

N

P(m, N, p)m0

N

Npq


Example: Suppose you observed m special events (success) in a sample of N events The measured probability (“efficiency”) for a special event to occur is:

What is the error (standard deviation) on the probability ("error on the efficiency"):

The sample size (N) should be as large as possible to reduce the uncertainty in the probability measurement. Let’s relate the above result to Lab 2 where we throw darts to measure the value of .If we inscribe a circle inside a square with side=s then the ratio of the area of the circle

to the rectangle is:

mN

m

N Npq

N N(1 )

N (1 )

N

4

)2/(

squareofarea

circleofarea2

2

2

2

s

s

s

r

So, if we throw darts at random at our rectangle then the probability () of a dart landing inside the circle is just the ratio of the two areas, /4. The we can determine using:

.The error in is related to the error in by:

N

)1(4

We can estimate how well we can measure by this method by assuming that = (3.14159…)/4:

4/using6.1)1(

4

NN

This formula “says” that to improve our estimate of by a factor of 10 we have to throw 100 (N) times as many darts! Clearly, this is an inefficient way to determine .

we will derive this later in the course

r

s


Example: Suppose a baseball player's batting average is 0.300 (3 for 10 on average).

Consider the case where the player either gets a hit or makes an out (forget about walks here!).

prob. for a hit: p = 0.30

prob. for "no hit”: q = 1 - p = 0.7

On average how many hits does the player get in 100 at bats?

= Np = 100·0.30 = 30 hits

What's the standard deviation for the number of hits in 100 at bats?

= (Npq)1/2 = (100·0.30·0.7)1/2 ≈ 4.6 hits

we expect ≈ 30 ± 5 hits per 100 at bats

Consider a game where the player bats 4 times:

probability of 0/4 = (0.7)4 = 24%

probability of 1/4 = [4!/(3!1!)](0.3)1(0.7)3 = 41%

probability of 2/4 = [4!/(2!2!)](0.3)2(0.7)2 = 26%

probability of 3/4 = [4!/(1!3!)](0.3)3(0.7)1 = 8%

probability of 4/4 = [4!/(0!4!)](0.3)4(0.7)0 = 1%

probability of getting at least one hit = 1 - P(0) = 1-0.24=76%

Pete Rose’s lifetimebatting average: 0.303


Poisson Probability Distribution The Poisson distribution is a widely used discrete probability distribution. Consider the following conditions:

p is very small and approaches 0 example: a 100 sided dice instead of a 6 sided dice, p = 1/100 instead of 1/6 example: a 1000 sided dice, p = 1/1000

N is very large and approaches ∞example: throwing 100 or 1000 dice instead of 2 dice

The product Np is finite Example: radioactive decay

Suppose we have 25 mg of an element very large number of atoms: N ≈ 1020 (avogadro’s number is large!)

Suppose the lifetime of this element = 1012 years ≈ 5x1019 seconds probability of a given nucleus to decay in one second is very small: p = 1/ = 2x10-20/sec BUT Np = 2/sec finite! The number of decays in a time interval is a Poisson process.

Poisson distribution can be derived by taking the appropriate limits of the binomial distribution

pNmNmN

m

mNm

epN

pNmNmNp

mNppq

NmN

mNmNNN

mN

N

qpmNm

NpNmP

!2

)(1

!2

)1)(()(1)1(

)!(

)!)(1()1(

)!(

!

)!(!

!),,(

22

radioactive decaynumber of Prussian soldiers kicked to death by horses per year !quality control, failure rate predictions

N>>m

...!3

)2)(1(

!2

)1(1)1(:using

32

aaaxaax

xax a

Siméon Denis PoissonJune 21, 1781-April 25, 1840


m is always an integer ≥ 0 does not have to be an integer

It is easy to show that: = Np = mean of a Poisson distribution2 = Np = = variance of a Poisson distribution

Radioactivity example with an average of 2 decays/sec: i) What’s the probability of zero decays in one second?

ii) What’s the probability of more than one decay in one second?

iii) Estimate the most probable number of decays/sec?

To solve this problem its convenient to maximize lnP(m, ) instead of P(m, ).

P(m,N, p)N m

m!pme pN

Let Np

P(m,)e m

m!

p(0,2)e 2 20

0!e 2 1

1e 2 0.135 13.5%

p(1,2)1 p(0,2) p(1,2)1 e 2 20

0! e 2 21

1!1 e 2 2e 2 0.594 59.4%

m

P(m,)m*0

ln P(m,)lne m

m!

m ln ln m!

The mean and variance ofa Poisson distribution are thesame number!

1!!

),(:note000

ee

me

m

emP

m

m

mm

m

mm

m


In order to handle the factorial when take the derivative we use Stirling's Approximation:

The most probable value for m is just the average of the distribution This is only approximate since Stirlings Approximation is only valid for large m.

Strictly speaking m can only take on integer values while is not restricted to be an integer.

If you observed m events in a “counting” experiment, the error on m is

ln m!m ln m m

m

ln P(m,) m

( m ln ln m!)

m

( m ln m ln mm)

ln ln m m1m1

0

m*

m

ln10!=15.10 10ln10-10=13.03 14%ln50!=148.48 50ln50-50=145.601.9%


0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Pro

babi

lity

0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0

m

binomialpoisson

N=10,p=0.1

0

0.1

0.2

0.3

0.4

0.5

Pro

babili

ty

0 1 2 3 4 5m

poissonbinomial

N=3, p=1/3

Comparison of Binomial and Poisson distributions with mean = 1

Not much differencebetween them!

For N large and fixed: Binomial Poisson

N N


Uniform distribution and Random NumbersWhat is a uniform probability distribution: p(x)?

p(x)=constant (c) for a x bp(x)=zero everywhere else

Therefore p(x1)dx1= p(x2)dx2 if dx1=dx2 equal intervals give equal probabilities

For a uniform distribution with a=0, b=1 we have p(x)=1

11)(1

0

1

0

1

0 cdxccdxdxxp

What is a random number generator ?A number picked at random from a uniform distribution with limits [0,1] All major computer languages (FORTRAN, C) come with a random number generator.

FORTRAN: RAN(iseed)The following FORTRAN program generates 5 random numbers: iseed=12345

do I=1,5 y=ran(iseed) type *, y enddo end

0.1985246 0.8978736 0.2382888 0.3679854 0.3817045

If we generate “a lot” of random numbersall equal intervals should contain the sameamount of numbers. For example: generate: 106 random numbers expect: 105 numbers [0.0, 0.1]

105 numbers [0.45, 0.55]

x

p(x)

1

10


1234567 FORTRAN statements start in column 7 implicit none character key real count(6), RAN integer i, dice, roll, seed seed = 432211111 10 continue do dice = 1, 6 count(dice) = 0 end do write(6,*)'How many rolls of dice :' read(5,*)roll do i = 1, roll dice = INT(1 + 6*RAN(seed)) count(dice) = count(dice) + 1 end do do i = 1, 6 write(6,*)i, count(i) end do write(6,*) ‘Hit <return> to continue or type q or Q', > ' and <return> to quit the program.' read(5,20)key20 format(a1) if (key.ne.'q'.and.key.ne.'Q') go to 10 end

A Fortran program to throw dice

real: variables with a decimal point, e.g. 321.45. 0.0567integer: variables without a decimal point, e.g. 5, 321

character: variables that are alphanumeric, e.g. A, a, osu

seed: an integer variable used by the random number generator RAN

A continue statement is a place holder. 10 is the statement numberThis is a “do loop”. It will repeat 6 times.

This question appears on the screen (unit 6)Here you answer the question (unit 5)

Another “do loop”.Here we are rolling the dice, using the random number generator.We keep track of the results in the array “count”.

Another “do loop”.Here we are writing the results to the monitor screen

Allows you to continue the program again or quit

If you don’t want to quit then we jump back up to line 10“end” tells the Fortran compiler that this is the end of the program

see thefortran tutorial


Throwing a dice using a computerWhen we throw an “honest” 6 sided dice we expect each number to appear 1/6 of the time.

To simulate this on the computer we want a program that generates the integers [1, 2, 3, 4, 5, 6] in a way that each number is equally likely.

DICE=INT(1+6*RAN(ISEED)) RAN(ISEED) gives a number [0,1], e.g. 0.33 6*RAN(ISEED) 1.98 (multiply by 6) 1+6*RAN(ISEED) 2.98 (add 1 to the number) INT(1+6*RAN(ISEED)) INT truncates 2.98 to an integer, 2

As a result of the above code DICE will be equal to “2”

We just rolled a “2”

How would we roll two dice with a computer?DICE1= INT(1+6*RAN(ISEED)) DICE2= INT(1+6*RAN(ISEED)) TWODI=DICE1+DICE2

The variable TWODI is an integer [2,12] which corresponds to the sum of the numbers on two independent rolls of the dice.

FORTRAN reads fromthe right to the left.

R.Kass/Sp07 P416 Lecture 2 1 Binomial and Poisson Probability Distributions Binomial Probability...

Documents

Transcript of R.Kass/Sp07 P416 Lecture 2 1 Binomial and Poisson Probability Distributions Binomial Probability...