1

Birth and death process

N(t) Depends on how fast arrivals or departures occur

Objective

N(t) = # of customersat time t.

λarrivals(births)

departures(deaths)

μ

))(Pr( itN

2

Behavior of the system λ>μ

λ<μ

Possible evolution of N(t)

Time 1 2 3 4 5 6 7 8 9 10 11

123

busy idleN(t)

3

General arrival and departure rates λn

Depends on the number of customers (n) in the system

Example

μn

Depends on the number of customers in the system

Example

MnMn

n ,0,

n

Orn

n

n

4

Changing the scale of a unit time

Number of arrivals/unit time Follows the Poisson distribution with rate λn

Inter-arrival time of successive arrivals is exponentially distributed

Average inter-arrival time = 1/ λn

What is the avg. # of customers arriving in dt?

Time

min/2/1min/16.030/30:

hourExampledtAverage

n

n

5

Probability of one arrival in dt dt so small

Number of arrivals in dt, X is a r.v. X=1 with probability p

X=0 with probability 1-p

Average number of arrivals in dt

Prob (having one arrival in dt) = λn dt

dtXEbutpppXE

n][,)1.(0.1][

dt

6

Probability of having 2 events in dt Departure rate in dt

μn dt

Arrival rate in dt λn dt

What is the probability Of having an (arrival+departure), (2 arrivals or departures)

222

222

2

)()()2Pr(

)()()2Pr(

)(.)11Pr(

dtdtdepartures

dtdtarrivals

dtdtdtdeparturearrival

nn

nn

nnnn

7

Probability distribution of N(t)

Pn (t) The probability of getting n customers by time t

The distribution of the # of customers in systemt+dtt

? nn-1: arrivaln+1: departuren: none of the above

)1)(()()()( 1111

dtdttPdttPdttPdttP

nnn

nnnnn

8

Differential equation monitoring evolution of # customers

?)())(()()()(

))(()()()()()1)(()()()(

1111'

1111

1111

tPtPtPtPtP

tPtPtPdt

tPdttPdtdttPdttPdttPdttP

n

nnnnnnnn

nnnnnnnnn

nnnnnnnn

These are solved Numerically using MATLAB

We will explore the cases Of pure death

And pure birth

9

Pure birth process In this case

μn =0, n >= 0

λn = λ, n >= 0

!)()(;)(

0,)()()(

0),(.)(

0,)()()(

0

1'

0'0

1'

nettPetP

ntPtPtP

ntPtP

ntPtPtP

tn

nt

nnn

nnn

Hence,

First order differential equation

10

11

Pure death process In this case

λn =0, n >= 0

μn = μ

)!()()(;)(

),(.)(

0,)()()(

0,)()()(

'

1'

1'

nMettPetP

MntPtP

ntPtPtP

ntPtPtP

tnM

nt

M

MM

nnn

nnn

12

Queuing system

Transient phase

Steady state Behavior is independent of t

Pn (t)

λ μ

nnt

PtP

)(lim

Pn (t)

t

transient Steady state

13

Differential equation: steady state analysis Limiting case

01

01

1111

1100

1111

'

0,)(

0,)(0

0)(

)(

limlim

PP

nPPPPP

nPPP

tP

PtP

nnnnnnn

nnnnnnn

nt

nnt

14

Solving the equations n=1

n=22200111 )( PPP

3311222 )( PPP

(1)

(1) =>

021

102

12

122211

22111111

PP

PPPP

PPPP

15

Pn

What about P0

1,......

021

110

0321

2102

23

233322

33222222

nPP

PP

PPPP

PPPP

n

nn

16

Normalization equation

1

1

1

0

0

021

100

1

00

10

1

1

1....

1.......

ni

n

i

i

n

i

n

P

PPP

ionNormilizatPPP

17

Conditional probability and conditional expectation: d.r.v. X and Y are discrete r.v.

Conditional probability mass function Of X given that Y=y

Conditional expectation of X given that Y=y

)(),(

)(),(

)|()|(|

ypyxp

yYPyYxXP

yYxXPyxp

Y

YX

x

yYxXPxyYXE )|(.]|[

18

Conditional probability and expectation: continuous r.v. If X and Y have a joint pdf fX,Y(x,y)

Then, the conditional probability density function Of X given that Y=y

The conditional expectation Of X given that Y=y

)(),()|(| yfyxfyxf

YYX

dxyxfxyYXE YX )|(.]|[ |

19

Computing expectations by conditioning Denote

E[X|Y]: function of the r.v. Y Whose value at Y=y is E[X|Y=y]

E[X|Y]: is itself a random variable Property of conditional expectation

if Y is a discrete r.v.

if Y is continuous with density fY (y) =>

]]|[[][ YXEEXE

y

yYPyYXEYXEEXE ][].|[]]|[[][

dyyfyYXEXE Y )(]|[][

(1)

(2)

(3)

20

Proof of equation when X and Y are discrete

][][

],[],[

][][],[

][].|[

][].|[]]|[[

XExXxP

yYxXPxyYxXxP

yYPyYPyYxXPx

yYPyYxXxP

yYPyYXEYXEE

x

x yy x

y x

y x

y

21

Problem 1 Sam will read

Either one chapter of his probability book or

One chapter of his history book

If the number of misprints in a chapter Of his probability book

is Poisson distributed with mean 2

Of his history book is Poisson distributed with mean 5

Assuming Sam equally likely to choose either book What is the expected number of misprints he comes across?

22

Solution

27)

21(2)

21(5

]2[].2|[]1[].1|[][_,2__,1

_

YPYXEYPYXEXEchosenyprobabilitchosenbookhistory

Y

mistakesnumberX

23

Problem 2 A miner is trapped in a mine containing three doors

First door leads to a tunnel that takes him to safety

After 2 hours of travel

Second door leads to a tunnel that returns him to the mine

After 3 hours of travel

Third door Leads to a tunnel that returns him to the mine

After 5 hours Assuming he is equally likely to choose any door

What is the expected length of time until he reaches safety?

24

Solution

10][][5][32(31][

][5]3|[];[3]2|[;2)1|[

])3|[]2|[]1|[(31

]3[]3|[]2[]2|[]1[]1|[][

____

XEXEXEXE

XEYXEXEYXEYXE

YXEYXEYXE

YPYXEYPYXEYPYXEXE

choseninitiallydoorYsafetyuntiltimeX

25

Computing probabilities by conditioning Let E denote an arbitrary event

X is a random variable defined by

It follows from the definition of X

tdoesnEif

occursEifX

'__,0__,1

continuousYdyyfyYEPEP

discreteYyYPyYEPEP

yYEPyYXEEPXE

Y

y

_,)()|()(

_,)().|()(

)3(&)2()|(]|[)(][

26

Problem 3 Suppose that the number of people

Who visit a yoga studio each day is a Poisson random variable with mean λ

Suppose further that each person who visit is, independently, female with probability p

Or male with probability 1-p

Find the joint probability That n women and m men visit the academy today

27

Solution Let

N1 denote the number of women, N2 the number of men Who visit the academy today

N= N1 +N2 : total number of people who visit Conditioning on N gives

Because P(N1=n,N2=m|N=i)=0 when i != n+m

0

2121 )().|,(),( iNPiNmNnNPmNnNP

)!().|,(

)().|,(),(

21

2121

mnemnNmNnNP

mnNPmnNmNnNPmNnNPmn

28

Solution (cont’d) Each of the n+m visit

is independently a woman with probability p

The conditional probability That n of them are women is

The binomial probability of n successes in n+m trials

!))1((

!)(

)!()1(),(

)1(

21

mpe

npe

mnepp

nmn

mNnNP

mp

np

mnmn

29

Solution: analysis

When each of a Poisson number of events is independently classified

As either being type 1 with probability p

Or type 2 with probability (1-p)

=> the numbers of type 1 and 2 events Are independent Poisson random variables

!))1(()(

!)(),()(

)1(2

0211

mpemNP

andnpemNnNPnNP

mp

m

np

30

Problem 4 At a party

N men take off their hats

The hats are then mixed up and Each man randomly selects one

A match occurs if a man selects his own hat

What is the probability of no matches?

31

Solution E = event that no matches occur

P(E) = Pn : explicit dependence on n

Start by conditioning Whether or not the first man selects his own hat

M: if he did, Mc : if he didn’t

P(E|Mc) Probability no matches when n-1 men select of n-1

That does not contain the hat of one of these men

nnMEPPMEP

MPMEPMPMEPEPP

cn

ccn

1)|(0)|(

);()|()()|()(

32

Solution (cont’d) P(E|Mc)

Either there are no matches and Extra man does not select the extra hat

=> Pn-1 (as if the extra hat belongs to this man)

Or there are no matches Extra man does select the extra hat

=> (1/n-1)xPn-2

21

21

1111)|(

nnn

nnc

Pn

PnnP

Pn

PMEP

33

Solution (cont’d) Pn is the probability of no matches

When n men select among their own hats => P1 =0 and P2 = ½

=>

!)1(....

!41

!31

!21

!41

!31

!21;

!31

!21

43

nP

PP

n

n

34

Problem 5: continuous random variables The probability density function of a non-negative

random variable X is given by

Compute the constant λ?

10.)(x

X exf

10110).10(1

)(1

0

10/

0

10/

x

xX

e

dxedxxf

35

Problem 6: continuous random variables Buses arrives at a specified stop at 15 min intervals

Starting at 7:00 AM They arrive at 7:00, 7:15, 7:30, 7:45

If the passenger arrives at the stop at a time Uniformly distributed between 7:00 and 7:30

Find the probability that he waits less than 5 min? Solution

Let X denote the number of minutes past 7 That the passenger arrives at the stop =>X is uniformly distributed over (0, 30)

31

301

301

)3025Pr()1510Pr(30

25

15

10

dxdx

XX

36

Problem 7: conditional probability

Suppose that p(x,y) the joint probability mass function of X and Y is given by P(0,0) = .4, P(0,1) = .2, P(1,0) = .1, P(1,1) = .3

Calculate the conditional probability mass function of X given Y = 1

)(),(

)(),(

)|()|(|

ypyxp

yYPyYxXP

yYxXPyxp

Y

YX

53

)1()1,1()1|1(,

52

)1()1,0()1|0(,

5.0)1,1()1,0()1,()1(

|

|

YYX

YYX

xY

PPPand

PPPhence

PPxPP

37

counting process A stochastic process {N(t), t>=0}

is said to be a counting process if N(t) represents the total number of events that occur by time t

N(t) must satisfy N(t) >= 0

N(t) is integer valued

If s < t, then N(s) <= N(t)

For s < t, N(s) – N(t) = # events in the interval (s,t]

Independent increments # of events in disjoint time intervals are independent

38

Poisson process The counting process {N(t), t>=0} is

Said to be a Poisson process having rate λ, if N(0) = 0

The process has independent increments

The # of events in any interval of length t is Poisson distributed with mean λt, that is

,...1,0,!)())()(( nntensNstNPn

t

39

Properties of the Poisson process Superposition property

If k independent Poisson processes A1, A2, …, An

Are combined into a single process A

=> A is still Poisson with rate Equal to the sum of individual λi of Ai

40

Properties of the Poisson process (cont’d) Decomposition property

Just the reverse process

“A” is a Poisson process split into n processes Using probability Pi

The other processes are Poisson With rate Pi.λ