Chapter 5: JOINT PROBABILITYDISTRIBUTIONS

Part 1: Sections 5-1.1 to 5-1.4

For both discrete and continuous random variables wewill discuss the following...

• Joint Distributions (for two or more r.v.’s)

• Marginal Distributions(computed from a joint distribution)

• Conditional Distributions(e.g. P (Y = y|X = x))

• Independence for r.v.’s X and Y

This is a good time to refresh yourmemory on double-integration. Wewill be using this skill in the upcom-ing lectures.

1

Recall a discrete probability distribution (orpmf ) for a single r.v. X with the example be-low...

x 0 1 2f (x) 0.50 0.20 0.30

Sometimes we’re simultaneously interested intwo or more variables in a random experiment.We’re looking for a relationship between the twovariables.

Examples for discrete r.v.’s

• Year in college vs. Number of credits taken

• Number of cigarettes smoked per day vs. Dayof the week

Examples for continuous r.v.’s

• Time when bus driver picks you up vs.Quantity of caffeine in bus driver’s system

• Dosage of a drug (ml) vs. Blood compoundmeasure (percentage)

2

In general, ifX and Y are two random variables,the probability distribution that defines their si-multaneous behavior is called a joint probabilitydistribution.

Shown here as a table for two discrete randomvariables, which gives P (X = x, Y = y).

x1 2 3

1 0 1/6 1/6y 2 1/6 0 1/6

3 1/6 1/6 0

Shown here as a graphic for two continuous ran-dom variables as fX,Y (x, y).

3

If X and Y are discrete, this distribution can bedescribed with a joint probability mass function.

If X and Y are continuous, this distribution canbe described with a joint probability density function.

• Example: Plastic covers for CDs(Discrete joint pmf)

Measurements for the length and width of arectangular plastic covers for CDs are roundedto the nearest mm (so they are discrete).

Let X denote the length andY denote the width.

The possible values of X are 129, 130, and131mm. The possible values of Y are 15 and16 mm (Thus, both X and Y are discrete).

4

There are 6 possible pairs (X, Y ).

We show the probability for each pair in thefollowing table:

x=length129 130 131

y=width 15 0.12 0.42 0.0616 0.08 0.28 0.04

The sum of all the probabilities is 1.0.

The combination with the highest probabil-ity is (130, 15).

The combination with the lowest probabilityis (131, 16).

The joint probability mass function is the func-tion fXY (x, y) = P (X = x, Y = y). Forexample, we have fXY (129, 15) = 0.12.

5

If we are given a joint probability distributionfor X and Y , we can obtain the individual prob-ability distribution for X or for Y (and theseare called the Marginal Probability Dis-tributions)...

• Example: Continuing plastic covers for CDs

Find the probability that a CD cover haslength of 129mm (i.e. X = 129).

x= length129 130 131

y=width 15 0.12 0.42 0.0616 0.08 0.28 0.04

P (X = 129) = P (X = 129 and Y = 15)+ P (X = 129 and Y = 16)

= 0.12 + 0.08 = 0.20

What is the probability distribution of X?

6

x= length129 130 131

y=width 15 0.12 0.42 0.0616 0.08 0.28 0.04

column totals 0.20 0.70 0.10

The probability distribution for X appearsin the column totals...

x 129 130 131fX(x) 0.20 0.70 0.10

∗ NOTE: We’ve used a subscript X in the probability

mass function of X , or fX(x), for clarification since

we’re considering more than one variable at a time

now.

7

We can do the same for the Y random variable:row

x= length totals129 130 131

y=width 15 0.12 0.42 0.06 0.6016 0.08 0.28 0.04 0.40

column totals 0.20 0.70 0.10 1

y 15 16fY (y) 0.60 0.40

Because the the probability mass functions forX and Y appear in the margins of the table(i.e. column and row totals), they are often re-ferred to as the Marginal Distributions forX and Y .

When there are two random variables of inter-est, we also use the term bivariate probabil-ity distribution or bivariate distributionto refer to the joint distribution.

8

• Joint Probability Mass FunctionThe joint probability mass function of the dis-crete random variables X and Y , denoted asfXY (x, y), satisfies

(1) fXY (x, y) ≥ 0

(2)∑x

∑y

fXY (x, y) = 1

(3) fXY (x, y) = P (X = x, Y = y)

For when the r.v.’s are discrete.

(Often shown with a 2-way table.)

x= length129 130 131

y=width 15 0.12 0.42 0.0616 0.08 0.28 0.04

9

•Marginal Probability Mass FunctionIf X and Y are discrete random variableswith joint probability mass function fXY (x, y),then the marginal probability mass functionsof X and Y are

fX(x) =∑y

fXY (x, y)

and

fY (y) =∑x

fXY (x, y)

where the sum for fX(x) is over all points inthe range of (X, Y ) for which X = x and thesum for fY (y) is over all points in the rangeof (X, Y ) for which Y = y.

We found the marginal distribution for X in theCD example as...

x 129 130 131fX(x) 0.20 0.70 0.10

10

HINT: When asked forE(X) or V (X) (i.e. val-ues related to only 1 of the 2 variables) but youare given a joint probability distribution, firstcalculate the marginal distribution fX(x) andwork it as we did before for the univariate case(i.e. for a single random variable).

• Example: BatteriesSuppose that 2 batteries are randomly cho-sen without replacement from the followinggroup of 12 batteries:

3 new4 used (working)5 defective

Let X denote the number of new batterieschosen.

Let Y denote the number of used batterieschosen.

11

a) Find fXY (x, y){i.e. the joint probability distribution}.

b) Find E(X).

ANS:

a) Though X can take on values 0, 1, and 2,and Y can take on values 0, 1, and 2, whenwe consider them jointly, X + Y ≤ 2. So,not all combinations of (X, Y ) are possible.

There are 6 possible cases...

CASE: no new, no used (so all defective)

fXY (0, 0) =

(52

)(

122

) = 10/66

12

CASE: no new, 1 used

fXY (0, 1) =

(41

)(51

)(

122

) = 20/66

CASE: no new, 2 used

fXY (0, 2) =

(42

)(

122

) = 6/66

CASE: 1 new, no used

fXY (1, 0) =

(31

)(51

)(

122

) = 15/66

13

CASE: 2 new, no used

fXY (2, 0) =

(32

)(

122

) = 3/66

CASE: 1 new, 1 used

fXY (1, 1) =

(31

)(41

)(

122

) = 12/66

The joint distribution for X and Y is...

x= number of new chosen0 1 2

y=number of 0 10/66 15/66 3/66used 1 20/66 12/66

chosen 2 6/66

There are 6 possible (X, Y ) pairs.And,

∑x∑y fXY (x, y) = 1.

14

b) Find E(X).

15

• Joint Probability Density FunctionA joint probability density function for thecontinuous random variable X and Y , de-noted as fXY (x, y), satisfies the followingproperties:

1. fXY (x, y) ≥ 0 for all x, y

2.∫∞−∞

∫∞−∞ fXY (x, y) dx dy = 1

3. For any region R of 2-D space

P ((X, Y ) ∈ R) =

∫ ∫RfXY (x, y) dx dy

For when the r.v.’s are continuous.

16

• Example: Movement of a particle

An article describes a model for the move-ment of a particle. Assume that a particlemoves within the region A bounded by the xaxis, the line x = 1, and the line y = x. Let(X, Y ) denote the position of the particle ata given time. The joint density of X and Yis given by

fXY (x, y) = 8xy for (x, y) ∈ A

a) Graphically show the region in the XYplane where fXY (x, y) is nonzero.

17

The probability density function fXY (x, y)is shown graphically below.

Without the information that fXY (x, y) = 0for (x, y) outside of A, we could plot the fullsurface, but the particle is only found in thegiven triangle A, so the joint probability den-sity function is shown on the right.

This gives a volume under the surface that isabove the region A equal to 1.

x

yf(x,y)

x

yf(x,y)

Not a pdf A pdf

18

b) Find P (0.5 < X < 1, 0 < Y < 0.5)

c) Find P (0 < X < 0.5, 0 < Y < 0.5)

19

d) Find P (0.5 < X < 1, 0.5 < Y < 1)

20

•Marginal Probability DensityFunction

If X and Y are continuous random variableswith joint probability density function fXY (x, y),then the marginal density functions forX andY are

fX(x) =

∫yfXY (x, y) dy

and

fY (y) =

∫xfXY (x, y) dx

where the first integral is over all points inthe range of (X, Y ) for which X = x, andthe second integral is over all points in therange of (X, Y ) for which Y = y.

HINT: E(X) and V (X) can be obtained byfirst calculating the marginal probability distri-bution of X , or fX(x).

21

• Example: Movement of a particle

An article describes a model for the move-ment of a particle. Assume that a particlemoves within the region A bounded by the xaxis, the line x = 1, and the line y = x. Let(X, Y ) denote the position of the particle ata given time. The joint density of X and Yis given by

fXY (x, y) = 8xy for (x, y) ∈ A

a) Find E(X)

22

Conditional Probability Distributions

Recall for events A and B,

P (A|B) =P (A∩B)P (B)

We now apply this conditioning to random vari-ables X and Y ...

Given random variables X and Y with jointprobability fXY (x, y), the conditionalprobability distribution of Y given X = x is

fY |x(y) =fXY (x,y)fX(x)

for fX(x) > 0.

The conditional probability can be statedas the joint probability over the marginalprobability.

Note: we can define fX|y(x) in a similar manner if we are

interested in that conditional distribution.

23

• Example: Continuing the plastic covers...

a) Find the probability that a CD cover hasa length of 130mm GIVEN the width is15mm.

ANS: P (X = 130|Y = 15) =P (X=130,Y =15)

P (Y =15)

= 0.420.60 = 0.70

b) Find the conditional distribution of Xgiven Y =15.

P (X = 129|Y = 15) = 0.12/0.60 = 0.20P (X = 130|Y = 15) = 0.42/0.60 = 0.70P (X = 131|Y = 15) = 0.06/0.60 = 0.10

24

Once you’re GIVEN that Y =15, you’re in a‘different space’.

For the subset of the covers with a width of15mm, how are the lengths (X) distributed.

The conditional distribution ofX given Y =15,or fX|Y =15(x):

x 129 130 131fX|Y =15(x) 0.20 0.70 0.10

The sum of these probabilities is 1, and thisis a legitimate probability distribution .

∗ NOTE: Again, we use the subscript X|Y for clarity

to denote that this is a conditional distribution.

25

A conditional probability distribution fY |x(y)has the following properties are satisfied:

• For discrete random variables (X,Y)

(1) fY |x(y) ≥ 0

(2)∑y

fY |x(y) = 1

(3) fY |x(y) = P (Y = y|X = x)

• For continuous random variables (X,Y)

1. fY |x(y) ≥ 0

2.∫∞−∞ fY |x(y) dy = 1

3. P (Y ∈ B|X = x) =∫B fY |x(y) dy

for any set B in the range of Y

26

•Conditional Mean and Variancefor DISCRETE random variables

The conditional mean of Y given X = x, de-noted as E(Y |x) or µY |x is

E(Y |x) =∑y

yfY |X(y) = µY |x

and the conditional variance of Y given X =x, denoted as V (Y |x) or σ2

Y |x is

V (Y |x) =∑y

(y − µY |x)2fY |X(y)

=∑y

y2fY |X(y)− µ2Y |x

= E(Y 2|x)− [E(Y |x)]2

= σ2Y |x

27

• Example: Continuing the plastic covers...

rowx=length totals129 130 131

y=width 15 0.12 0.42 0.06 0.6016 0.08 0.28 0.04 0.40

column totals 0.20 0.70 0.10 1

a) Find the E(Y |X = 129) andV (Y |X = 129).

ANS:We need the conditional distribution first...

y 15 16fY |X=129(y)

28

29

•Conditional Mean and Variancefor CONTINUOUS random variables

The conditional mean of Y given X = x,denoted as E(Y |x) or µY |x, is

E(Y |x) =∫yfY |x(y) dy

and the conditional variance of Y given X =x, denoted as V (Y |x) or σ2

Y |x, is

V (Y |x) =∫

(y − µY |x)2fY |x(y) dy

=∫y2fY |x(y) dy − µ2

Y |x

30

• Example 1: Conditional distribution

Suppose (X, Y ) has a probability density func-tion...

fXY (x, y) = x+ y for 0 < x < 1, 0 < y < 1

a) Find fY |x(y).

b) Show∫∞−∞ fY |x(y)dy = 1.

31

a)

32

b)

One more...c) What is the conditional mean of Y given

X = 0.5?

ANS:

First get fY |X=0.5(y)

fY |x(y) =x + y

x + 0.5for 0 < x < 1 and 0 < y < 1

fY |X=0.5(y) =0.5 + y

0.5 + 0.5= 0.5 + y for 0 < y < 1

E(Y |X = 0.5) =

∫ 1

0y(0.5 + y) dy =

7

1233

Independence

As we saw earlier, sometimes, knowledge of oneevent does not give us any information on theprobability of another event.

Previously, we stated that if A and B were in-dependent, then

P (A|B) = P (A).

In the framework of probability distributions,if X and Y are independent random variables,then fY |X(y) = fY (y).

34

• IndependenceFor random variables X and Y , if any of thefollowing properties is true, the others arealso true, and X and Y are independent.

(1) fXY (x, y) = fX(x)fY (y) for all x and y

(2) fY |x(y) = fY (y)

for all x and y with fX(x) > 0

(3) fX|y(x) = fX(x)

for all x and y with fY (y) > 0

(4) P (X ∈ A, Y ∈ B) = P (X ∈ A) · P (Y ∈ B)

for any sets A and B in the range of X and Y.

Notice how (1) leads to (2):

fY |x(y) =fXY (x,y)fX(x)

=fX(x)fY (y)fX(x)

= fY (y)

35

• Example 1: (discrete)Continuing the battery example

Two batteries were chosen without replace-ment.

Let X denote the number of new batterieschosen.

Let Y denote the number of used batterieschosen.

x= number of new chosen0 1 2

y=number 0 10/66 15/66 3/66of used 1 20/66 12/66chosen 2 6/66

a) Without doing any calculations, can youtell whether X and Y are independent?

36

• Example 2: (discrete)Independent random variables

Consider the random variables X and Y ,which both can take on values of 0 and 1.

rowx totals

0 1y 0 0.08 0.02 0.10

1 0.72 0.18 0.90column totals 0.80 0.20 1

a) Are X and Y independent?

y 0 1fY |X=0(y)

37

y 0 1fY |X=1(y)

Does fY |x(y) = fY (y) for all x & y?

Does fXY (x, y) = fX(x)fY (y) for all x & y?

rowx totals

0 1y 0 0.08 0.02 0.10

1 0.72 0.18 0.90column totals 0.80 0.20 1

i.e. Does P (X = x, Y = y)= P (X = x) · P (Y = y)?

38

• Example 3: (continuous)Dimensions of machined parts (Example 5-12).

Let X and Y denote the lengths of two di-mensions of a machined part.

X and Y are independent and measured inmillimeters (you’re given independence here).

X ∼ N(10.5, 0.0025)Y ∼ N(3.2, 0.0036)

a) FindP (10.4 < X < 10.6, 3.15 < Y < 3.25).

39