Lecture Note 5 Discrete Random Variables and Probability Distributions ©

Lecture Note 5Lecture Note 5

Discrete Random Discrete Random Variables and Variables and

Probability Probability DistributionsDistributions

©

Random VariablesRandom Variables

A random variablerandom variable is a variable that takes on numerical values determined by the outcome of a random experiment.

Discrete Random VariablesDiscrete Random Variables

A random variable is discrete discrete if it can take on no more than a countable number of values.

Discrete Random VariablesDiscrete Random Variables(Examples)(Examples)

1. The outcome of a roll of a die. 2. The number of defective items

in a sample of twenty items taken from a large shipment.

Continuous Random Continuous Random VariablesVariables

A random variable is continuous continuous if it can take any value in an interval.

Continuous Random Continuous Random VariablesVariables

(Examples)(Examples)

1. Heights of students in a class2. Average temperature in November3. The income in a year for a family.4. The amount of oil imported into the

U.S. in a particular month.

Discrete Random Discrete Random VariablesVariables

We will study the fundamental concepts of discrete random variables, namely (1) Probability Distribution Function and (2) Cumulative Probability Function.

First, we will learn the notation of the Probability Distribution Function

Understanding the notation Understanding the notation for the probability for the probability

distribution functiondistribution function

Suppose you are about to roll a die. Then, the number you would get after the roll is a discrete random variable. Let X denote this random variable.

Then this random variable X can take 6 possible outcomes from S={1, 2, 3, 4, 5, 6}.

Understanding the notation for the Understanding the notation for the probability distribution function, probability distribution function,

contdcontd

Then, the probability that the random variable X takes the value 1 (i.e., the probability that you get 1 after rolling the die) is 1/6. This can be conveniently expressed using the following notation:

P(1)=P(X ＝ 1)=1/6

Similarly, we have P(2)=P(X=2)=1/6 P(3)=P(X=3)=1/6 . . P(6)=P(X=6)=1/6


contdcontd We can generalize the notation from

the previous slides to other discrete random variables. Let X be any discrete random variable which takes k possible values from S={x1, x2, …, xk}

Conventionally, we use capital letter

X to denotes the random variable, and small letter x to denote the possible values that X can take.


contdcontd

Now, let x be any values from S={x1, x2, …, xk}

Then, the probability that the random variable X takes the value x can be written as:

P(x)=P(X=x)

This notation is what it is called “Probability Distribution Function”. The next slide re-iterates this definition.

Probability Distribution Probability Distribution FunctionFunction

The probability distribution probability distribution function,function, P(x), of a discrete random variable expresses the probability that X takes the value x, as a function of x. That is

. of valuesallfor ),()( xxXPxP


-Exercise 1--Exercise 1-

Ex 1-1: Suppose you roll a die. Let X be the random variable for this experiment. Find the probability distribution function P(x) by completing the table in the Excel Sheet “Probability Distribution Function Exercise”.

Ex 1-2: Plot the probability distribution function.



Consider the following game. You toss a coin. If you get the heads, you receive \100. If you gets the tails, you receive nothing.

The payoff for this game is a discrete random variable. Let X denote this random variable. Find the probability distribution of this random variable by completing the table in “Probability Distribution Function Exercise”.

Required Properties of Required Properties of Probability Distribution Probability Distribution

Functions of Discrete Random Functions of Discrete Random VariablesVariables

Let X be a discrete random variable with probability distribution function, P(x). Then

i. P(x) 0 for any value of xii. The individual probabilities sum to 1; that is

Where the notation indicates summation over all possible values x.

x

xP 1)(

Cumulative Probability Cumulative Probability FunctionFunction

The cumulative probability function,cumulative probability function, F(x0), of a random variable X expresses the probability that X does not exceed the value x0, as a function of x0. That is

Where the function is evaluated at all values x0

)()( 00 xXPxF

Cumulative Probability Cumulative Probability FunctionFunction

-Example 3--Example 3-

3-1: Let X be the random variable for a roll of a die. Find F(1), F(2), F(3), to F(6). Use the Excel sheet, `Probability distribution function exercise’.

3-2: Plot F(x) and x.

Derived Relationship Between Derived Relationship Between Probability Function and Cumulative Probability Function and Cumulative

Probability FunctionProbability Function

Let X be a random variable with probability function P(x) and cumulative probability function F(x0). Then it can be shown that

Where the notation implies that summation is over all possible values x that are less than or equal to x0.

0

)()( 0xx

xPxF

Expected ValueExpected Value Expected value is a similar concept as the

average. More specifically, it is the weighted average with the weight given by the probability of each outcome.

For example, consider the following game. You toss a coin. If you get the heads, you receive \100. If you get the tails, you receive nothing.

Then the expected payoff for this game is \100×(probability of getting \100) + \0×(probability of getting \0) = \50 More formally, expected value is defined

as follows: See next slide.

Expected ValueExpected Value

The expected value, E(X),expected value, E(X), of a discrete random variable X is defined

Where the notation indicates that summation extends over all possible values x.The expected value of a random variable is called its meanmean and is denoted xx.

x

xxPXE )()(

Understanding the notation Understanding the notation of the expected valueof the expected value

Let X be the random variable which takes a value in S={x1, x2,..,xk}.

Then the expected value of X is computed as follows. See next slide

Random variable X

P(x) xP(x)

x1 P(x1) x1P(x1)

x2 P(x2) x2P(x2)

: : :

xk P(xk) xkP(xk)

μX=

k

ikk xPx

1

)(

Expected Value ExerciseExpected Value Exercise-Exercise 4--Exercise 4-

Let X be the random variable that shows the number of house purchase contracts that a real estate agent can achieve in a month. The probability distribution of X is given by the following.

Compute the expected number of house purchase contracts that the real estate agent can achieve in a month.

x P(x)

0 0.05

1 0.07

2 0.12

3 0.2

4 0.3

5 0.2

6 0.06

Expected Value: Functions Expected Value: Functions of Random Variablesof Random Variables

Let X be a discrete random variable with probability function P(x) and let g(X) be some function of X. Then the expected value, E[g(X)], of that function is defined as

x

xPxgXgE )()()]([

Expected Value: Functions Expected Value: Functions of Random Variablesof Random Variables

-Exercise 5--Exercise 5- Continue using the real estate agent

example. Suppose that the monthly salary for the real estate agent is given by:

g(x) =$1000+ $1500*x

Ex 5-1: Find the expected monthly salary of the agent. Use the “Probability Distribution Function Exercise” Excel Sheet.

Variance and Standard Variance and Standard DeviationDeviation

Let X be a discrete random variable. The expectation of the squared deviation about the mean, (X - x)2, is called the variancevariance, denoted 2

x and is given by

The standard deviationstandard deviation, x , is the positive square root of the variance.

Often we use the Var(X) to denote the variance and SD(X) to denote the standard deviation

x

xxx xPxXE )()()( 222

Understanding the Understanding the procedure to compute the procedure to compute the

variancevariance

The table in the next slide summarizes the procedure to compute variance.

Variable X

P(x) xP(x) (x- μX)2P(x)

x1 P(x1) x1P(x1) (x1-μX)2P(x1)

x2 P(x2) x2P(x2) (x2-μX)2P(x2)

: : : :

xk P(xk) xkP(xk) (xk-μX)2P(xk)

Sum=μX

Sum=σ2X

Variance and Standard Variance and Standard DeviationDeviation

-Exercise 6--Exercise 6- Continue using the real estate

agent example. Using the table in the “Probability Distribution Function Exercises” Excel sheet, answer the following questions:

Ex 6-1 Compute the varianceEx 6-2 Compute the standard

deviation

x P(x)

0 0.05

1 0.07

2 0.12

3 0.2

4 0.3

5 0.2

6 0.06

Summary of Properties for Summary of Properties for Linear Function of a Random Linear Function of a Random

VariableVariable

Let X be a random variable with mean x , and variance 2

x ; and let a and b be any constant fixed numbers. Define the random variable Y = a + bX. Then, the meanmean and variancevariance of Y are

and

so that the standard deviation of Ystandard deviation of Y is

XY babXaE )(

XY bbXaVar 222 )(

XY b

Linear Function of a Random Linear Function of a Random VariableVariable


Continue using the example of the real estate agent from exercise 4, 5, and 6.

Suppose that the monthly salary for this agent, Y, is determined in the following method:

Y=$1000+ $1500*X

Ex 7-1: Compute E(Y) using the formula in the previous slide.

Ex 7-2: Compute the variance and standard deviation of Y

Standardization of a Random Standardization of a Random VariableVariable

Let X be a random variable with mean X and standard deviation X. Define a new random variable Z as

Then

E(Z)=0

Var(Z)=1

X

XXZ

Exercise 8Exercise 8

Continue using the real estate agent example. Let X be the random variable for the possible number of contracts the agent can achieve in a month.

Standardize X using the formula in the previous slide, and verify that it has mean zero and variance 1.

Reviews of Combinatorics Reviews of Combinatorics and Common discrete and Common discrete

distributionsdistributions

Review of Combinatorics1. Number of orderings2. Number of combinations3. Number of sequences of x Successes in n Trials

Common discrete distributions1. Bernoulli Distribution2. Binomial Distribution

Stock Price Movement Stock Price Movement ExampleExample

To motivate the study of combinatorics and the discrete distributions, let us consider a simple example of a stock price movement. The example will utilize combinatorics and distributions.


The price of Stock A today is $10. At the end of each month the price of the stock A either goes up by the factor of 1.2 with probability 0.4, or goes down by the factor of 0.9 with probability 0.6.

This means that, if the price of the stock goes up this month, then the price of the stock in the second month will be $10×1.2 =$12. If the price of the stock goes up this month, and goes down in the second month, the price of the stock in the third month will be $10×1.2×0.9 = $10.8.

Suppose that you make the following contract with a stock broker:

“You have the right to purchase a share of stock A at the price equal to $37.61 at the beginning of the 13th month.

Then, if the actual market price of stock A is higher than $37.61, you purchase the stock and immediately sell it to make a positive profit (ignoring fees associated with the contract and transaction fee). If the actual price is lower than 37.61, you do not have to exercise the right.

Then consider the following question: See next slide

Stock Price Movement Example, Contd

Stock Price Movement Example, Contd

Question What is the probability that you make

some strictly positive profit from this contract (ignoring all the fees associated with the contract and buying and selling of the share)?

Answer to this question requires tools such as combinatorics and Binomial distribution.

In the following slides, we will review combinatorics and some common distributions. After studying these concepts, we will come back to the question.

Review of CombinatoricsReview of Combinatorics-Number of orderings--Number of orderings-

You have n cards numbered from 1 to n. The number of ways you can order this card is given by

n!=n×(n-1) ×(n-2) ×…×3×2×1

n! reads n factorial 0! is defined to be 1.

Number of ordering Number of ordering exampleexample

There are 5 cards numbered from 1 to 5. What is the number of ways you can order this card?

Review of CombinatoricsReview of Combinatorics-Combinations--Combinations-

Suppose there are n cards numbered from 1 to n. If you take x cards out of n cards, the number of possible combinations of the cards is given by

)!(!

!

xnx

nC nx

Cnx reads n choose x.

Where n!=n×(n-1) ×(n-2) ×…×3×2×1

Combinations Combinations -Example--Example-

A personnel officer has eight candidate to fill four similar positions. What is the total number of possible combinations of four candidates chosen from eight?

Review of combinatoricsReview of combinatorics- sequences of x Successes - sequences of x Successes

in n Traials-in n Traials-

Consider a random trial where there are only two outcomes, “Success” or “Failure”.

Then, if you make n independent trials, the number of sequences that contains exactly x successes is equal to Cn

x.

Sequences of SuccessesSequences of Successes-Exercise--Exercise-

Consider tossing a coin 10 times. Find the number of ways the head appears exactly 4 times.

Number of sequencesNumber of sequences-Exercise--Exercise-

Suppose that the figure below is a road map of a certain area. If you start from point A and walk to point B, how many possible routes can you take? (Suppose you do not walk back.)

A

B

Bernouli TrialBernouli Trial

This is a random experiment with two possible outcomes, “success” or “failure”, where the probabilities are given by

P(Success) = P(Failure) = 1-

Bernouli Random Trial with success probability .

Bernouli Random VariableBernouli Random Variable Consider a Bernouli Trial with

P(Success)= . Define a random variable X in the following way.

X=1 if the Bernouli random trial turn out to be success

X=0 if the Bernouli random trial turns out to be a failure.

Then, X is called a Bernouli Random Variable

with success probability .

Bernouli DistributionBernouli Distribution

Consider a Bernouli random variable X with success probability . Then the probability distribution function for X is called Bernouli Distribution with success probability . This is given by

P(0)=(1- ) and P(1)=

Bernouli Distribution Bernouli Distribution -Example--Example-

Consider the following game. You toss a coin. If you get the heads, you receive $1. If you get the tails, you receive nothing. Let X be the random variable for the payoff of this game.

X has the Bernouli distribution with success probability 0.5.

Mean and Variance of a Mean and Variance of a Bernoulli Random VariableBernoulli Random Variable

The meanmean is:

And the variancevariance is:

)1()1)(0()()( xPxXEX

X

)1()1()1()0(

)()(])[(

22

222

X

XXX xPxXE

Binomial DistributionBinomial Distribution-Example--Example-

I use an example to illustrate Binominal Distribution.

You inspect a production line by randomly checking the items. It is known that 5% of the products from this production line are defect items.

Suppose you randomly choose 4 items from the production line. Then, the number of defect items you would find is a discrete random variable.

Let X be the number of defect items in the 4 randomly picked items. What is the probability that exactly 2 of them are defect items?

To answer to this question, first, consider the possible sequences that 2 defect items are picked.

Binomial Probability

-Examples, Contd-

The number of possible sequences you pick 2 defect items is given in the table. D denotes the defect item, and G denotes the good item.

Note that the number of sequences is given by C4

2=6.

The possible sequences you pick 2 defect items

Sequence 1

DDGG

Sequence 2

DGDG

Sequence 3

GDDG

Sequence 4

GGDD

Sequence 5

GDGD

Sequence 6

DGGD

Now, what is the probability of getting Sequence 1? That is, what is the probability that you first get D, second you get D, third you get G and fourth you get G?

The answer to this question: since the probability of getting D is 0.05, and the probability of getting G is (1-0.05)=0.95, the probability of getting this sequence is 0.05*0.05*0.95*0.95 =0.002256.

In fact, the probability of getting any of the 6 sequences is the same; since each sequence contains exactly 2 defective items and 2 good items, the probability of getting each sequence is 0.0520.952=0.002256.

Since there are 6 sequences, the probability that you get exactly 2 defect items is given by (0.002256)×6=0.013536.

This is equal to C42(0.05)2(1-0.05)2

We can generalize this problem. Consider a production line. The probability that an item from this production line is a defect item is .

Suppose that you pick n products from this production line. Then, the probability that the number of defect items is exactly x is given by

xnxnxCxP )1()(

•This probability distribution function is called Binomial Distribution with “success” probability . Next slide summarizes the binomial distribution.

Binomial DistributionBinomial Distribution

Suppose that a random experiment can result in two possible mutually exclusive and collectively exhaustive outcomes, “success” and “failure,” and that is the probability of a success resulting in a single trial. If n independent trials are carried out, the distribution of the resulting number of successes “x” is called the binomial binomial distributiondistribution. Its probability distribution function for the binomial random variable X = x is:

P(x successes in n independent trials)=

for x = 0, 1, 2 . . . , n

)()1()!(!

!)( xnx

xnx

nxP

Mean and Variance of a Mean and Variance of a Binomial Probability Binomial Probability

DistributionDistributionLet X be the number of successes in n independent trials, each with probability of success . The x follows a binomial distribution with meanmean,

and variancevariance,

nXEX )(

)1(])[( 22 nXEX

Binomial ProbabilitiesBinomial Probabilities- An Example - An Example

A sales person randomly visits houses to sell a certain product. He believes that for each visit, the probability of making a sale is 0.40.

If the sales person visits 5 houses, what is the probability that he makes at least 3 sales?

AnswerAnswer

Let X be the random number for the number of sales. Then,

P(At leaset 3 sale) = P(X ≥ 3) = P(X =3) + P(X = 4)+P(X=5)

01024.0)6.0()4.0(5)!-(55!

5!P(5)5)P(X

0768.0)6.0()4.0(4)!-(54!

5! P(4) 4)P(X

2304.0)6.0()4.0(3)!-(53!

5! P(3) 3)P(X

)55(5

)45(4

)35(3

P(makes at least 3 sales)=P(3)+P(4)+P(5)

=0.2304+0.0768+0.01024=0.31744

Computing Binomial Computing Binomial Distribution using ExcelDistribution using Excel

Let n be the total number of trials. Let x be the number of success, and be the success probability. The Excel function to compute binomial probabilities is

P(X=x)= BINOMDIST(x, n, , FALSE) P(X≤x)=BINOMDIST(x,n, , TRUE)

Note that if you put FALSE at the end, it computes the binomial probability distribution. If you put TRUE at the end, it computes the cumulative binomial distribution.

Binomial DistributionBinomial Distribution-Exercise--Exercise-

Open “Binomial Distribution Exercise”

Find the Binomial Distribution Function for n=50 and =0.3. Then, graph the Binomial Probability Distribution.

Binomial Distribution Binomial Distribution Function with n=50 and Function with n=50 and

=0.3.=0.3.

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

P(x

): P

roba

bilit

y D

istr

ibut

ion

x: Number of successes

Binimial Distribution With n=50 and Success prob=0.3


Now, we come back to the Stock Price Movement example that we saw at the beginning.

First, take a look at how the stock price may move in the first three months. See the next slide

This month$10

Second month $12

$9

Third monthx1 = $14.4

x2 = $10.8

x3 = $8.1

As it can be seen, there are three possible values for the stock price in the third month. Now answer the questions in the next slide.

Stock price movement for the first three months

Q1: In the 13th month, how many possible values for the stock price are there?

Q2. Show that the 4th highest price is $37.61Q3. What is the probability that the stock

price at the beginning of 13th month is equal to the 4th highest price, $37.61?

Q4. Remember the contract. The contract is “You have the right to purchase a share of the stock A in the 13th month at the price equal to $37.61”. Then, what is the probability that you get some positive profit from this contract (ignoring all the fees associated with buying and selling the stock).

Joint Probability FunctionsJoint Probability Functions

Consider two stocks, Stock A and Stock B. Let X denote the random variable for the return of stock A. Let Y denote the random variable for the return of Stock B.

Suppose that X takes 4 possible values 0, 0.05, 0.1, and 0.15.

Further, suppose that Y takes 4 possible values, 0, 0.05, 0.1 , and 0.15.

The joint probabilities of X and Y are given in the table in the next slide.

Joint Probability FunctionJoint Probability Function-Example--Example-

Y return

X return 0 0.05 0.1 0.15

0 0.02 0.1 0.05 0.01

0.05 0.03 0.1 0.1 0.05

0.1 0.1 0.12 0.01 0.01

0.15 0.15 0.13 0.01 0.01

Joint Probability FunctionsJoint Probability Functions

Let X and Y be a pair of discrete random variables. Their joint probability functionjoint probability function expresses the probability that X takes the specific value x and simultaneously Y takes the value y, as a function of x and y. The notation used is P(x, y) so,

)(),( yYxXPyxP

Marginal Probability Marginal Probability FunctionsFunctions

Let X and Y be a pair of jointly distributed random variables. In this context the probability function of the random variable X is called its marginal probability functionmarginal probability function and is obtained by summing the joint probabilities over all possible values; that is,

Similarly, the marginal probability functionmarginal probability function of the random variable Y is

y

yxPxP ),()(

x

yxPyP ),()(

ExerciseExercise

Open “Joint Probability Exercise”. This sheet contains the joint probability example of stock A return (X) and Stock B return (Y) in the previous example.

Find the Marginal distribution.

Conditional Probability Conditional Probability FunctionsFunctions

Let X and Y be a pair of jointly distributed discrete random variables. The conditional probability functionconditional probability function of the random variable Y, given that the random variable X takes the value x, expresses the probability that Y takes the value y, as a function of y, when the value x is specified for X. This is denoted P(y|x), and so by the definition of conditional probability:

Similarly, the conditional probability functionconditional probability function of X, given Y = y is:

)(

),()|(

xP

yxPxyP

)(

),()|(

yP

yxPyxP

ExerciseExercise

Continue using the same example. Compute the following conditional probabilities.

P(X=0|Y=0.1)P(Y=0.15|X=0.1)

Independence of Jointly Independence of Jointly Distributed Random Distributed Random

VariablesVariables

The jointly distributed random variables X and Y are said to be independentindependent if and only if their joint probability function is the product of their marginal probability functions, that is, if and only if

And k random variables are independent if and only if

y. and x valuesof pairs possible allfor )()(),( yPxPyxP

)()()(),,,( 2121 kk xPxPxPxxxP

ExerciseExercise

Using the example in the “Joint Distribution Exercise” check to see if X and Y are statistically independent.

DefinitionDefinition

Let X and Y be jointly distributed discrete random variables with joint probability distribution function P(x,y). Then, the expectation of a function of X and Y, g(X,Y) is defined as

),(),()],([ yxPyxgYXgEyx

CovarianceCovariance

Let X be a random variable with mean X , and let Y be a random variable with mean, Y . The expected value of (X - X )(Y - Y ) is called the covariance covariance between X and Y, denoted Cov(X, Y).For discrete random variables

An equivalent expression is

x y

yxYX yxPyxYXEYXCov ),())(()])([(),(

x y

yxyx yxxyPXYEYXCov ),()(),(

CorrelationCorrelation

Let X and Y be jointly distributed random variables. The correlation between X and Y is:

YX

YXCovYXCorr

),(

),(

Covariance and Statistical Covariance and Statistical IndependenceIndependence

If two random variables are statistically statistically independentindependent, the covariance between them is 0. However, the converse is not necessarily true.

ExerciseExercise

Consider the following joint distribution of X and Y

Value of Y

3 4

Value of X

1 0.1 0.2

2 0.3 0.4

Exercise: Cov(X,Y), Var(X), Var(Y) and CORR(X,Y).

Portfolio AnalysisPortfolio Analysis

The random variable X is the price for stock A and the random variable Y is the price for stock B. The market value, W, of the portfolio is given by the linear function,

Where, a, is the number of shares of stock A and, b, is the number of shares of stock B.

bYaXW

Portfolio AnalysisPortfolio Analysis

The mean value for Wmean value for W is,

The variance for Wvariance for W is,

or using the correlation,

YX

W

ba

bYaXEWE

][][

),(222222 YXabCovba YXW

YXYXW YXabCorrba ),(222222

ExerciseExercise Consider Stock A and Stock B. Let X

and Y denote the market price of stock A and B, respectively. It is known that E(X)=10, E(Y)=20, Var(X)=2, Var(Y)=4, and Cov(X,Y)=‒1. Now consider the following portfolio.

W=5X +4Y.

Find E(W), Var(W).

Lecture Note 5 Discrete Random Variables and Probability Distributions ©

Documents

Transcript of Lecture Note 5 Discrete Random Variables and Probability Distributions ©