The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Chapter 4. Discrete Probability Distributions

Section 4.11: Markov Chains

Jiaping Wang

Department of Mathematical Science

03/18/2013, Monday

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Outline

Introduction

Formal Definition and Formulas

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Part 1. Introduction

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Consider a system that can be in any of a finite number of states. Assume that the system moves from state to state according to some prescribed probability law. For example, the states represent whether the economy is in a bull market, a bear market or a recession during a given week. From week to week, the states will transit each other, eg., from bull market to bear market, or from recession to bear market with some probabilities.From the figure, if we denote Xi as the state in i-th week, S1=Bull market,S2=Bear market, S3=Recession, then we can haveP(Xi=S1|Xi-1=S2)=0.15=p21, P(Xi=S2|Xi-1=S2)=0.80=p22,P(Xi=S2|Xi-1=S1)=0.075=p12, P(Xi=S1|Xi-1=S1)=0.90 =p11,P(Xi=S3|Xi-1=S1)=0.025 =p13,P(Xi=S3|Xi-1=S2)=0.05 =p23,P(Xi=S1|Xi-1=S3)=0.25 =p31,P(Xi=S2|Xi-1=S3)=0.25 =p32,P(Xi=S3|Xi-1=S3)=0.50 =p33,

An Example

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

So based on the transit probabilities given above, we can construct a transition matrix

P=

We can find P11+P12+P13=1, P21+P22+P23=1 and P31+P32+P33=1.

Once we are given this transition system, we can obtain all possible probabilities between the states in any given time.

Transition Matrix

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Part 2. Formal Definitions

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Markov Chain

Formally, a Markov chain is a random process with the Markov property. Often, the term "Markov chain" is used to mean a Markov process which has a discrete (finite or countable) state-space. Let Xi denote the state of the system at time point I and let the possible states be denoted by S1, S2, …, Sm for a finite integer m. We assume that

P(Xi=Sk|Xi-1=Sj)=pjk, where pjk is the transition probability from Sj to Skwhich is independent of i. The event (Xi=Sk|Xi-1=Sj) is assumed to be independent of past history of the process (Markov property).

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Denote p(0)=[p1(0),p2

(0),…, pm(0)] with pk

(0)=P(X0=Sk) where X0 denote the starting state of the system. The p(n)=[p1

(n),…,pm(n)] is

the probability after n steps. So we have

p(1)=p(0)P, and p(n)=p(n-1)P

Where P is the transition matrix.

If P is regular, then there is a limit π=(π1,…, πm)=limn∞p(n), which satisfies

π= πP.

Basic Operation: Probability after n steps

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Example 4.29

A supermarket stocks three brands of coffee – A, B and C – and customers switch from brand to brand according to the transition matrix

Where S1 corresponds to a purchase of brand A, S2 to brand B, S3 to brand C; that is ¾ of the customers buying brand A also buy brand A the next time they purchase coffee, whereas ¼ of these customers switch to brand B.

1. Find the probability that a customer who buys brand A today will again purchase brand A 2 weeks from today, assuming that he or she purchase coffee once a week.

2. In the long run, what fractions of customers purchase the respective brands?

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Solution

Answer:1. Assuming that the customer is chosen at random, his or her transition

probabilities are given by P. The given information indicates that P(0)=(1, 0, 0); that is, the customer starts with a purchase of brand A. Then we have

p(1)=p(0)P=(3/4, ¼, 0)Gives the probabilities for the next week’s purchase. The probabilities for the

two weeks from now are given byp(2)=p(1)P=(9/16, 17/18, 1/12).

That is, the chance of the customer to purchase brand A 2 weeks from now is 9/16.

2. The answer to the long-run frequency ratio is given by π, which is the stationary distribution. The equation π = πP yields

π1=(3/4) π1+(1/4) π2 π2=(1/4) π1+(2/3) π2+(1/4) π3 π3=(1/3) π2+(1/2) π3 π1+ π2+ π3=1

Solve them to π=(2/7, 3/7, 2/7).

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Chapter 5. Continuous Probability Distributions

Section 5.1: Continuous Random Variables and Their Probability Distributions

Jiaping Wang

Department of Mathematical Science

03/18/2013, Monday

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Outline

Introduction

Density Function

Distribution Function

More Examples

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Part 1. Introduction

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

An Experiment

Measuring the life length X of 50 batteries of certain type.

0.406 0.685 4.778 1.725 8.523

2.343 1.401 1.507 0.294 2.230

0.538 0.234 4.025 3.323 2.920

5.088 1.458 1.064 0.774 0.761

5.587 0.517 3.246 2.330 1.064

2.563 0.511 3.246 2.330 1.064

6.523 0.225 1.514 3.214 3.810

3.334 2.325 0.333 7.554 0.968

3.491 2.921 1.624 0.334 4.490

1.267 1.702 2.634 1.849 0.186

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Part 2. Density Function

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Definition 5.1

A random variable X is said to be continuous if there is a function f(x), called probability density function, such that

Notice that P(X=a)=P(a ≤ X ≤ a)=0.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Example 5.1

The random variable X of the life lengths of batteries discussed earlier is associated with a probability density function of the form

Find the probability that the life of a particular battery of this type is less than 200 or greater than 400 hours.

Answer: Let A be the event that X less than 2 (Hundreds of hours), B be that X isGreater than 4, also A and B are mutually exclusive, so P(AU B)=P(A)+P(B)=P(X≤2)+P(X≥4) =(1-e-1)+(e-2)=0.767

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Example 5.2

Refer to Example 5.1. Find the probability that a battery of this type lasts more than 3 (Hundreds of hours), given that it already has been in use for more than 2 (Hundreds of hours).

Answer: Let A be the event that X has been in use for more than 2, ie., A={X>2}, let B Denote the event that X lasts more than 3, ie., B={X>3}, so we are interested inThe new event C={B|A}={X>3|X>2}, so the probability of P(C) is

P(X>3|X>2)=P(X>3, X>2)/P(X>2)=P(X>3)/P(X>2)=e-3/2/e-1=0.606

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Part 3. Distribution Function

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Definition 5.2

The distribution function for a random variable X is defined as

F(b)=P(X ≤ b).If X is continuous with probability density function f(x), then

Notice that F’(x)=f(x).For example, we are given

Thus,

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Properties

The cumulative distribution function F(x) of the continuous random variable X follows the properties:

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Example 5.3

The distribution function of the random variable X, the time (in months) from the diagnosis age until death for one population of patients with AIDS, is as follows:

F(x)=1-e-0.03x1.2, x>0.

1. Find the probability that a randomly selected person from this population survives at least 12 months.

2. Find the probability density function of X.

Answer: 1. P(X ≥ 12) = 1- P(X≤12) = 1-F(12) = 1-(1- e-0.03(12)1.2) = 1-0.45=0.55.

2. f(x)=F’(x)=0.036x0.2e-0.03x1.2, x>0; and 0, otherwise.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Part 3. More Examples

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Additional Example 1

Determine the value of c so that the following function is a density function

Answer: Based on the requirement for a density function ∫f(x)dx=1, we have

∫c/(x+1)3dx=1 c=1/[∫1/(x+1)3dx ]= 2.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Additional Example 2

Let X have the density function

If P(X>1)=7/8, find the value of θ.

Answer: first find F(b)=(b/θ)3, then we have P(X>1)=1-P(X≤1)=1-F(1)=1-(1/θ)3=7/8,

So from 1-(1/θ)3=7/8, we have θ=2.