Chapter 5: JOINT PROBABILITY DISTRIBUTIONS...

of 17/17
Chapter 5: JOINT PROBABILITY DISTRIBUTIONS Part 2: Covariance and Correlation Section 5-2 Consider the joint probability distribution f XY (x, y ). Is there a relationship between X and Y ? If so, what kind? If you’re given information on X , does it give you information on the distribution of Y ? (Think of a conditional distribution). Or are they inde- pendent? 1
  • date post

    25-Apr-2018
  • Category

    Documents

  • view

    221
  • download

    6

Embed Size (px)

Transcript of Chapter 5: JOINT PROBABILITY DISTRIBUTIONS...

  • Chapter 5: JOINT PROBABILITYDISTRIBUTIONS

    Part 2: Covariance and CorrelationSection 5-2

    Consider the joint probability distribution fXY (x, y).

    Is there a relationship between X and Y ? If so,what kind?

    If youre given information on X , does it giveyou information on the distribution of Y ? (Thinkof a conditional distribution). Or are they inde-pendent?

    1

  • Below is a different joint probability distribu-tion for X and Y .

    Does there seem to be a relationship between Xand Y ? Are they independent?

    If youre given information on X , does it giveyou information on the distribution of Y ?

    How would you describe the relationship?

    Is it stronger than the relationship on the pre-vious page? Do you know MORE about Y fora given X?

    2

  • Below is a joint probability distribution for anindependent X and Y .

    This picture is the give-awaythat theyre independent.

    Does there seem to be a relationship between Xand Y ?

    If youre given information on X , does it giveyou information on the distribution of Y ?

    3

  • Covariance

    When two random variables are being consid-ered simultaneously, it is useful to describe howthey relate to each other, or how they vary to-gether.

    A common measure of the relationship betweentwo random variables is the covariance.

    CovarianceThe covariance between the random variablesX and Y , denoted as cov(X, Y ), or XY , is

    XY = E[(X E(X))(Y E(Y ))]

    = E[(X X)(Y Y )]

    = E(XY ) E(X)E(Y )

    = E(XY ) XY4

  • To calculate covariance, we need to find the ex-pected value of a function of X and Y . Thisis done similarly to how it was done in the uni-variate case...

    For X, Y discrete,E[h(x, y)] =

    xy h(x, y)fXY (x, y)

    For X, Y continuous,E[h(x, y)] =

    h(x, y)fXY (x, y)dxdy

    Covariance (i.e. XY ) is an expected value ofa function of X and Y over the (X, Y ) space,if X and Y are continuous we can write

    XY =

    (xX)(yY )fXY (x, y) dx dy

    To compute covariance, youll probably use...

    XY = E(XY ) E(X)E(Y )

    5

  • When does the covariance have a positive value?

    In the integration were conceptually puttingweight on values of (x X)(y Y ).

    What regions of (X, Y ) space has...(x X)(y Y ) > 0?

    Both X and Y are above their means. Both X and Y are below their means. Values along a line of positive slope.

    A distribution that puts high probability on theseregions will have a positive covariance.

    6

  • When does the covariance have a negativevalue?

    In the integration were conceptually puttingweight on values of (x X)(y Y ).

    What regions of (X, Y ) space has...(x X)(y Y ) < 0?

    X is above its mean, and Y is below its mean. Y is above its mean, andX is below its mean. Values along a line of negative slope.

    A distribution that puts high probability on theseregions will have a negative covariance.

    7

  • Covariance is a measure of the linear relationshipbetween X and Y .

    If there is a non-linear relationship between Xand Y (such as a quadratic relationship), thecovariance may not be sensitive to this.

    When does the covariance have a zero value?

    This can happen in a number of situations, buttheres one situation that is of large interest...when X and Y are independent...

    When X and Y are independent, XY = 0.

    8

  • If X and Y are independent, then...

    XY =

    (x X)(y Y )fXY (x,y) dx dy

    =

    (x X)(y Y )fX(x)fY (y) dx dy

    =

    (

    (x X)fX(x)dx)(

    (y Y )fY (y)dy)

    =

    (

    xfX(x)dx X)(

    yfY (y)dy Y)

    = (E(X) X) (E(Y ) Y )= (X X) (Y Y )= 0

    This does NOT mean... If covariance=0, thenX and Y are independent.

    We can find cases to the contrary of the abovestatement, like when there is a strong quadraticrelationship between X and Y (so theyre notindependent), but you can still get XY = 0.

    Remember that covariance specifically looks fora linear relationship.

    9

  • When X and Y are independent, XY = 0.

    For this distribution showing independence,there is equal weight along the positive and neg-ative diagonals.

    10

  • A couple comments...

    You can also define covariance for discrete Xand Y :

    XY = E[(X X)(Y Y )]

    =xy(x X)(y Y )fXY (x, y)

    And recall that you can get the expectedvalue of any function of X and Y :

    E[h(X, Y )] =

    h(x, y)fXY (x, y) dx dy

    or

    E[h(X, Y )] =xy h(x, y)fXY (x, y)

    11

  • Correlation

    Covariance is a measure of the linear relationshipbetween two variables, but perhaps a more com-mon and more easily interpretable measure iscorrelation.

    CorrelationThe correlation (or correlation coefficient) be-tween random variables X and Y , denotedas XY , is

    XY =cov(X, Y )V (X)V (Y )

    =XYXY

    .

    Notice that the numerator is the covariance,but its now been scaled according to thestandard deviation of X and Y (which areboth > 0), were just scaling the covariance.

    NOTE: Covariance and correlation will havethe same sign (positive or negative).

    12

  • Correlation lies in [1, 1], in other words,

    1 XY +1

    Correlation is a unitless (or dimensionless)quantity.

    Correlation...

    1 XY +1 If X and Y have a strong positive linear re-

    lation XY is near +1.

    If X and Y have a strong negative linearrelation XY is near 1. If X and Y have a non-zero correlation, they

    are said to be correlated.

    Correlation is a measure of linear relation-ship.

    If X and Y are independent, XY = 0.

    13

  • Example: Recall the particle movement model

    An article describes a model for the move-ment of a particle. Assume that a particlemoves within the region A bounded by the xaxis, the line x = 1, and the line y = x. Let(X, Y ) denote the position of the particle ata given time. The joint density of X and Yis given by

    fXY (x, y) = 8xy for (x, y) A

    a) Find cov(X, Y )

    ANS: Earlier, we found E(X) = 45 ...

    14

  • 15

  • Example: Book problem 5-43 p. 179.

    The joint probability distribution is

    x -1 0 0 1y 0 -1 1 0fXY 0.25 0.25 0.25 0.25

    Show that the correlation between X and Yis zero, but X and Y are not independent.

    16

  • 17