of 17/17
Chapter 5: JOINT PROBABILITY DISTRIBUTIONS Part 2: Covariance and Correlation Section 5-2 Consider the joint probability distribution f XY (x, y ). Is there a relationship between X and Y ? If so, what kind? If you’re given information on X , does it give you information on the distribution of Y ? (Think of a conditional distribution). Or are they inde- pendent? 1
• date post

25-Apr-2018
• Category

## Documents

• view

221

6

Embed Size (px)

### Transcript of Chapter 5: JOINT PROBABILITY DISTRIBUTIONS...

• Chapter 5: JOINT PROBABILITYDISTRIBUTIONS

Part 2: Covariance and CorrelationSection 5-2

Consider the joint probability distribution fXY (x, y).

Is there a relationship between X and Y ? If so,what kind?

If youre given information on X , does it giveyou information on the distribution of Y ? (Thinkof a conditional distribution). Or are they inde-pendent?

1

• Below is a different joint probability distribu-tion for X and Y .

Does there seem to be a relationship between Xand Y ? Are they independent?

If youre given information on X , does it giveyou information on the distribution of Y ?

How would you describe the relationship?

Is it stronger than the relationship on the pre-vious page? Do you know MORE about Y fora given X?

2

• Below is a joint probability distribution for anindependent X and Y .

This picture is the give-awaythat theyre independent.

Does there seem to be a relationship between Xand Y ?

If youre given information on X , does it giveyou information on the distribution of Y ?

3

• Covariance

When two random variables are being consid-ered simultaneously, it is useful to describe howthey relate to each other, or how they vary to-gether.

A common measure of the relationship betweentwo random variables is the covariance.

CovarianceThe covariance between the random variablesX and Y , denoted as cov(X, Y ), or XY , is

XY = E[(X E(X))(Y E(Y ))]

= E[(X X)(Y Y )]

= E(XY ) E(X)E(Y )

= E(XY ) XY4

• To calculate covariance, we need to find the ex-pected value of a function of X and Y . Thisis done similarly to how it was done in the uni-variate case...

For X, Y discrete,E[h(x, y)] =

xy h(x, y)fXY (x, y)

For X, Y continuous,E[h(x, y)] =

h(x, y)fXY (x, y)dxdy

Covariance (i.e. XY ) is an expected value ofa function of X and Y over the (X, Y ) space,if X and Y are continuous we can write

XY =

(xX)(yY )fXY (x, y) dx dy

To compute covariance, youll probably use...

XY = E(XY ) E(X)E(Y )

5

• When does the covariance have a positive value?

In the integration were conceptually puttingweight on values of (x X)(y Y ).

What regions of (X, Y ) space has...(x X)(y Y ) > 0?

Both X and Y are above their means. Both X and Y are below their means. Values along a line of positive slope.

A distribution that puts high probability on theseregions will have a positive covariance.

6

• When does the covariance have a negativevalue?

In the integration were conceptually puttingweight on values of (x X)(y Y ).

What regions of (X, Y ) space has...(x X)(y Y ) < 0?

X is above its mean, and Y is below its mean. Y is above its mean, andX is below its mean. Values along a line of negative slope.

A distribution that puts high probability on theseregions will have a negative covariance.

7

• Covariance is a measure of the linear relationshipbetween X and Y .

If there is a non-linear relationship between Xand Y (such as a quadratic relationship), thecovariance may not be sensitive to this.

When does the covariance have a zero value?

This can happen in a number of situations, buttheres one situation that is of large interest...when X and Y are independent...

When X and Y are independent, XY = 0.

8

• If X and Y are independent, then...

XY =

(x X)(y Y )fXY (x,y) dx dy

=

(x X)(y Y )fX(x)fY (y) dx dy

=

(

(x X)fX(x)dx)(

(y Y )fY (y)dy)

=

(

xfX(x)dx X)(

yfY (y)dy Y)

= (E(X) X) (E(Y ) Y )= (X X) (Y Y )= 0

This does NOT mean... If covariance=0, thenX and Y are independent.

We can find cases to the contrary of the abovestatement, like when there is a strong quadraticrelationship between X and Y (so theyre notindependent), but you can still get XY = 0.

Remember that covariance specifically looks fora linear relationship.

9

• When X and Y are independent, XY = 0.

For this distribution showing independence,there is equal weight along the positive and neg-ative diagonals.

10

You can also define covariance for discrete Xand Y :

XY = E[(X X)(Y Y )]

=xy(x X)(y Y )fXY (x, y)

And recall that you can get the expectedvalue of any function of X and Y :

E[h(X, Y )] =

h(x, y)fXY (x, y) dx dy

or

E[h(X, Y )] =xy h(x, y)fXY (x, y)

11

• Correlation

Covariance is a measure of the linear relationshipbetween two variables, but perhaps a more com-mon and more easily interpretable measure iscorrelation.

CorrelationThe correlation (or correlation coefficient) be-tween random variables X and Y , denotedas XY , is

XY =cov(X, Y )V (X)V (Y )

=XYXY

.

Notice that the numerator is the covariance,but its now been scaled according to thestandard deviation of X and Y (which areboth > 0), were just scaling the covariance.

NOTE: Covariance and correlation will havethe same sign (positive or negative).

12

• Correlation lies in [1, 1], in other words,

1 XY +1

Correlation is a unitless (or dimensionless)quantity.

Correlation...

1 XY +1 If X and Y have a strong positive linear re-

lation XY is near +1.

If X and Y have a strong negative linearrelation XY is near 1. If X and Y have a non-zero correlation, they

are said to be correlated.

Correlation is a measure of linear relation-ship.

If X and Y are independent, XY = 0.

13

• Example: Recall the particle movement model

An article describes a model for the move-ment of a particle. Assume that a particlemoves within the region A bounded by the xaxis, the line x = 1, and the line y = x. Let(X, Y ) denote the position of the particle ata given time. The joint density of X and Yis given by

fXY (x, y) = 8xy for (x, y) A

a) Find cov(X, Y )

ANS: Earlier, we found E(X) = 45 ...

14

• 15

• Example: Book problem 5-43 p. 179.

The joint probability distribution is

x -1 0 0 1y 0 -1 1 0fXY 0.25 0.25 0.25 0.25

Show that the correlation between X and Yis zero, but X and Y are not independent.

16

• 17