Discrete Distributions

Random Variable

A random variable X is a function that maps the possible outcomes of an experiment to real numbers.

That is X: C --> R, where C is the set of all outcomes of an experiment and R is the set of real numbers.

The space of X is the set of real numbers S = {x: X(c)= x, }

Cc

An Example of Random Variable

If we toss a coin one time, then there are two possible outcomes, namely “head up” and “tail up”.

We can define a random variable X that maps “head up” to 1 and “tail up” to 0.

We also can define a random variable Y that maps “head up” to 0 and “tail up” to 1.

The spaces of both random variables X and Y are {0,1}.

Further Illustration of Random Variables

A random variable corresponds to a quantitative interpretation of the outcomes of an experiment.

For example, a company offers its employees a drawing in its yearend party. A computer will randomly select an employee for the first prize of $100,000 based on the employees’ ID number, which ranges from 1 to 100.

In addition, the computer will randomly select two more employees for the second and third prizes of $50,000 and $10,000, respectively.

Assume that each employee can receive only one award and the drawing starts with the third prize and ends with the first prize.

Then, there are totally 100 × 99 × 98 = 970200 possible outcomes.

To Edward, whose employee ID number is 10, the random variable of his interest is as follows:X(<10, *, *>) = 10,000X(<*, 10, *>) = 50,000X(<*, *, 10>) = 100,000X(all other outcomes) = 0

To Grace, whose employee ID number is 30, the random variable of her interest is as follows:Y(<30, *, *>) = 10,000Y(<*, 30, *>) = 50,000Y(<*, *, 30>) = 100,000Y(all other outcomes) = 0

The outcome spaces of random variables X and Y are identical. However, X and Y map some outcomes to different real numbers.

The spaces of X and Y are also identical and both are {0, 10000, 50000, 100000}.

The probability functions of X and Y are also equal.Prob(X=10,000) = Prob(Y=10,000) = 0.01Prob(X=50,000) = Prob(Y=50,000)

= 0.01Prob(X=100,000) = Prob(Y=100,000)

= 0.01Prob(X=0) = Prob(Y=0) = 0.97

The expected values of X and Y are equal to

E[X] = E[Y]

= 10,000 * 0.01 + 50,000 * 0.01

+ 100,000 * 0.01

= 1600.

Discrete Random Variables

Given a random variable X, let S denote the space of X.

If S is a finite or countable infinite set, then X is said to be a discrete random variable.

Countable Infinite

A set is said to be countable infinite, if it contains infinite number of elements and there exists a one-to-one mapping between each element of the set and the positive integers.

Examples of Countable / Uncountable Infinite

The set of integer numbers is countable.

The set of fractional numbers is countable.

The set of real numbers is uncountable.

Probability Mass Function

The probability mass function (p.m.f.) of a discrete random variable X is defined to be

. variablerandomby tomapped are

thatoutcomes all contains where

),(

Xk

Q

qkXkP

k

QqX

k

ProbProb

In the previous example of drawing,

0.01. 9899100

1

),,10(

000,10000,10

10,10,,10

10,10,,10

jijiji

jijiji

X

ji

XP

Prob

Prob

In fact,the p.m.f. of a random variable is defined on a set of events of the experiment conducted.

In the previous drawing example, the set of outcomes that are mapped to 10,000 by X is an event.

Furthermore, in the previous drawing example, random variables X and Y map some outcomes to different real numbers. However, X and Y have the same distribution, i.e. the p.m.f. of X and the p.m.f. of Y are equal. More precisely,

100000}. 50000, {0,10000,every for

)()(

k

kPkP YX

Properties of the Probability Mass Function

The p.m.f. of a random variable X satisfies the following three properties:

.3

12

.0 then finite, is If

01

S A , wherexPA

. xP

xPS

X. space ofS : the , xx P

AxjX

SxiX

X

X

j

i

Prob

Probability Distribution Function

For a random variable X, we define its probability distribution function F as

tXProbtFX

Properties of a Probability Distribution Function

Any function that satisfies these conditions above can be a distribution function.

. , . 3

. 0lim . 2

. 1lim .1

twiftFwF

tF

tF

XX

Xt

Xt

An Example of the Probability Distribution Function of a Discrete Random Variable Assume that we toss a 4-sided die twice. Then, we have 16 possible outcomes ：

4434241443332313

4232221241312111

,,,,,,,,,,,,,,,

,,,,,,,,,,,,,,,,

Let random variable X be the sum of the outcome.Then,

.8

5

16

4

16

3

16

2

16

15Prob5

. 16

18Prob

16

27Prob ,

16

36Prob

16

45Prob ,

16

34Prob

16

23Prob ,

16

12Prob

XF

X

XX

X X

X X

X

Operations of Random Variables

Let X and Y be two random variables defined on the same outcome space of an experiment.

Then, we can define a new random variable Z=f(X,Y).

For example, in the example of drawing, if Edward and Grace are husband and wife, then we can define a new random variable Z=X+Y.

We haveX(<30, 10, *>) = 50,000Y(<30, 10, *>) = 10,000Z(<30, 10, *>) = 60,000

Function of Random Variables

Let X be a random variable and G be a function. Then, random variable Y=G（ X） maps an outcome ν in the outcome space of X to value G（ X（ ν ）） .

With respect to the probability distribution functions, if G（ X） is monotonically increasing, one-to-one mapping, then

tGFtGX

tXGtYtF

X

Y

11Prob

ProbProb

An Example of Functions of Random Variables

Let random variable X be the sum of two tosses of a 4-sided die and Y=X2.

Then,

.8

3

16

6234

.44Prob

16Prob16Prob16 2

XXX

X

Y

PPP

FX

XYF

Expected Value of a Discrete Random Variable

Let X be a discrete random variable and S be its space. Then, the expected value of X is

μ is a widely used symbol for expected value.

iSx

iXCz

xxPzXzobXEi

)()(Pr

Expected Value of a Function of a Random Variable

Let X be a random variable and G be a function. Then, the expected value of random variable is equal to XGY

Sx

iXi

i

xPxG

Expected Value of a Function of a Random Variable Proof ：

SxjjX

Syj

yxGthatsuch

xallj

iSy

i

iSy

iY

j

i

ij

j

i

i

xGxP

xGxX

yyY

YofspacetheisSwhereyyPYE

.

Prob

Prob

. ,

'

'

'

'

For example, let X correspond to the outcome of tossing a die once. Then, Px(1)=Px(2)=Px(3)=Px(4)=Px(5)=Px(6)=1/6. and E[X]=3.5

If we are concerned about the difference between the observed outcome and the mean. And define Y=|X-E[X]|, then PY(1/2)=1/3, PY(3/2)=1/3, PY(5/2)=1/3.

.2

3

6

9

6

1)5.25.15.05.05.15.2(

6

1|5.3|)(|][|

hand,other On the

.2

3

3

1

2

9

3

1

2

5

3

1

2

3

3

1

2

1][

Therefore,

ii x

ix

ixi xxPXEx

YE

Theorems about the Expected Value

（ a） If c is a constant,

（ b） If c is a constant and g

is a function,

（ c） If c1 and c2 are constants

and g1 and g2 are functions,

then

.ccE

XgcEXcgE

.22112211 XgEcXgEcXgcXgcE

Proof of （ a ）： Trivial. Proof of （ b ）：

XgcE

xPxgc

X. p.m.f of the

is xX and P of

es the spac,where S ixPxcgXcgE

SxiXi

X

SxiXi

i

i


Proof of （ c ）：

An extension of （ c ）

.

2211

2211

22112211

XgEcXgEc

xPxgcxPxgc

xPxgcxgcXgcXgcE

iXSx

iiXSx

i

iXSx

ii

ii

i

.11

k

iii

k

iii XgEcXgcE


Variance of a Discrete Random Variable

The variance of a random variable is defined to be and is typically denoted by σ2.

For a discrete random variable X,

σ is normally called the standard deviation.

2XE

.

2

2

22

22

222

XE

XEXE

XXEXEXVar

Let X be a random variable with mean μX and variance σX

2. Let Y= aX+b, where a and b are constants.

Then,

.

222222

2

2

XXX

X

y

X

aXEaXaE

babaXE

YEYVar

babXaEbaXEYE

Variance of a Discrete Random Variable

Variance of a Random Variable

The variance of a random variable measures the deviation of its distribution from the mean.

For example, in one drawing, Robert has 0.1% of chance to win $100,000, while in another drawing, he has 0.01% of chance to win $1,000,000.

The expected amounts of award in these two drawings are equal.

0.001 * 100000 = 100

0.0001 * 1000000 = 100 However, their variances are

different.

0.001 * (100000 – 100)2

+ 0.999 * (0 – 100)2 = 9,990,000

0.0001 * (1000000 – 100)2

+ 0.999 * (0 – 100)2 = 99,990,000

In many distributions, the mean and variance together uniquely determine the parameters of the random variables.

The Bernoulli Experiment and Distribution

A Bernoulli experiment is a random experiment, the outcome of which can be classified in one of two mutually exclusive and exhaustive ways, say, success and failure.

A sequence of Bernoulli trials occurs when a Bernoulli experiment is performed several independent times, so that the probability of success, say p, remains the same from trial to trial.

Let X be a Bernoulli random variable. The p.m.f of X can be written as

where k= 0 or 1 and p is the probability of success.

The expected value of X is

The variance of X is

,1 1 kkX ppkP

1

0

1 .1k

kk ppkp

1

0

12 .11k

kk pppppk

The Bernoulli Distribution

The Binomial Distribution

Let X be the random variable corresponding to the number of successes in a sequence of Bernoulli trials.

Then,

where n is the number of Bernoulli trials and p is the probability of success in one trial.

X is said to have a binomial distribution and is normally denoted by b（ n , p） .

,1Prob knknkX ppCkXkP

Example of the Binomial Distribution

Assume that Tiger and Whale are the two teams that enter the Championship series of the professional basket ball league. Based on prior records, Tiger has a 60% chance of beating Whale in a single game. Larry, who is a fan of Tiger, makes a bet with Peter, who is a fan of Whale. According to their agreement, Larry will pay Peter $1000, should Whale win the 5-game series. In order to make a fair bet, how much should Peter pay Larry, if Tiger wins the series?

The probability that Tiger wins the series is

.465

)6826.01(*10006826.0*

6826.0)6.0()4.0()6.0()4.0()6.0( 555

454

2353

Z

Z

CCC

If the championship series consists of 3 games, then what is the probability that Tiger win the series?

6826.0648.0

)6.0()4.0()6.0( 333

232

CC

The Moment-Generating Function

Let X be a discrete random variable with p.m.f and space S. If there is a positive number h such that

exists and is finite for -h＜ t＜h, then the function of t defined by

is called the moment-generating function of X. and often abbreviated as m.g.f.

xPX

Sx

iXtxtX

i

i xPeeE

tXeEtM

Let X and Y be two discrete random variables with the same space S.

If then the probability mass functions of X and Y are equal.

Insight of the argument above ： Assume that S=｛ s1,s2, …, sk｝ contains only positive integers.

Then, we have

Therefore, , i.e. X and Y have the same p.m.f.

, tYtX eEeE

. ...

...21

21

21

21

k

k

tskY

tsY

tsY

tskX

tsX

tsX

esPesPesP

esPesPesP

iYiX sPsP


Let be the m.g.f of a discrete random variable X.

Furthermore,

In particular,

.

Sx

iXtxk

iKX

K

i

i xPexdt

tMd

Sx

kiX

kiK

XK

i

XExPxdt

Md..

0

tM X

.00 02

2

XXXXX MMandM


The Moment-Generating Function of the Binomial Distribution

Let X be b（ n , p） .

are both difficult to compute.

n

k

knk

n

k

knknk

n

k

knk

n

k

knknk

ppknk

kn

ppCkXE

ppknk

n

ppkCXE

0

0

22

0

0

1!!1

!

1

1!!1

!

1

On the other hand, we can easily

derive the m.g.f. of a binomial

distribution.

.11

1

0

0

nn

k

tknktnk

n

k

knknk

tktXX

ppeppeC

ppCeeEtM

.10

0

111

1

2

122

1

nppnnM

npM

ppenpepeppenntM

peppentM

X

X

ntttntX

tntX

Therefore,

.-pnp

pnnpnppn

MMσ

npM

XXX

XX

1

00

0

22222

22

The Moment-Generating Function of the Binomial Distribution

The Poisson Process

A Poisson process models the number of times that a particular type of events occur during a time interval.

The Poisson process is based on the following 3 assumptions ：

（ 1） The numbers of event occurrences in non-overlapping intervals are independent.

（ 2 ） Prob（ one occurrence between times t and ） =

0lim

t

tt .t

（ 3 ） Prob（ two occurrences between times t and ） = 0.

λ is the only parameter of the Poisson process.

One example of the Poisson process is to model the number of Web accesses that a Web server receives between 8 AM and 9 AM.

The Poisson Process

0lim

t

tt

The Basis of the Assumptions of the Poisson Process

Assume that an ideal random number generator generates λ numbers in [0, 1].

If we divide [0, 1] evenly into n subintervals,then the probability that there is exactly one of the number generated in [0, 1/n] is

.1

11

11

11

1

nnnnC

The Basis of the Assumptions of the Poisson Process

The probability that there are exactly two of the numbers generated in [0, 1/n] is

Let .Then,

nt /1

.2

111

2

1limt0,in soccurrence twoProblim

.1

1limt0,in occurrence oneProblim

22

2

0

1

0

tn

t

tn

t

nt

nt

.

11

2

111

12

2

22

2

nnnnC

The Poisson Distribution

Assume that we are concerned about a Poisson process with parameter λ and want to count the number of event occurrences during one time interval.

We can divide the time interval evenly into n subintervals as the following figure shows.1/n

Time=1Time=0

The probability that the event occurs k times during the time interval is

Since and ,

the final result is .

k

n

k

nk

n

kk

n

k

n

k

n

knknk

n

n

nk

n

nnk

n

n

nnknk

n

nnC

1

1

!lim

1

1

!lim

1

1

!!

!lim1lim

11lim

k

n n

,1lim

e

n

n

n

ek

k

!


We say that a random variable X has a Poisson distribution, if

By the Maclaurin’s series,

we have

Therefore,

.!

ek

kPk

X

0

.!

1

k

k

ke

0 0

.1!k k

k

X eek

ekP


The moment-generating function of a random variable with the Poisson distribution is

112

1

1

0

0

!

!

tt

t

tt

etetX

etX

ee

k

kt

k

kktXt

X

eeeetM

eetM

eeek

ee

ek

eeEtM


Therefore,

Therefore, λ is the average rate of event occurrence per unit of time.

Let Y be the random variable corresponding to the number of event occurrences during a time interval of length t. Then,

.

!t

k

Y ek

tkP


.

00 and 0

22

22

XXXXX MMM


The probability that the event occurs k times during a time interval of length t is

tk

knk

n

knknk

n

ek

t

n

t

n

t

k

t

n

t

n

tC

!

11!

lim

1lim

Joint Distributions

Joint Probability Mass Function

Let X and Y be two discrete random variables defined on the same outcome set. The probability that X=x and Y=y is denoted by PX,Y(x, y)= Prob(X=x,Y=y) and is called the joint probability mass function(joint p.m.f) of X and Y. PX,Y(x, y) satisfies the the following 3 properties:

S.S ofsubset a isA where

,,,Pr )3(

1, )2(

1,0 )1(

,,

,,

,

AyxYX

SyxYX

YX

yxPAYXob

yxP

yxP

Example of Joint Distributions Assume that a supermarket collected the

following statistics of customers’ purchasing behavior:

Purchasing

Wine

Not Purchasing

Wine

Male 45 255

Female 70 630

Purchasing

Juice

Not Purchasing

Juice

Male 60 240

Female 210 490

Example of Joint Distributions

Let random variable M correspond to whether a customer is male, random variable W correspond to whether a customer purchases wine, random variable J correspond to whether a customer purchases juice.

The joint p.m.f of M and W is

W

M

PMW (1,1) = 0.045

PMW (1,0) = 0.255

PMW (0,1) = 0.07

PMW (0,0) = 0.63

The joint p.m.f of M and J is

W

M

PMW (1,1) = 0.06

PMW (1,0) = 0.24

PMW (0,1) = 0.21

PMW (0,0) = 0.49

Marginal Probability Mass Function

Let PXY(x,y) be the joint p.m.f. of discrete random variables X and Y.

is called the marginal p.m.f of X. Similarly,

is called the marginal p.m.f. of Y.

j

j

yiXY

yiX

yxP

yYxXobxXobxP

),(

,PrPr

ix

iYXY yxPyP ,,

Note that we can always create a common outcome set for any two or more random variables. For example, let X and Y correspond to the outcomes of the first and second tosses of a coin, respectively. Then, the outcome set of X is {head up, tail up} and the outcome set of Y is also {head up, tail up}. The common outcome set of X and Y is {(head up,head up),(head up,tail up),(tail up,head up),(tail up,tail up)}.

More on Joint Probability Mass Function

Independent Random Variables

Two discrete random variables X and Y are said to be independent if and only if for all possible combination of x and y

Otherwise, X and Y are said to be dependent.

.,, yPxPyxP YXYX

Example of Independent Random Variables Assume that a supermarket collected the


Purchasing

soft drinks

Not purchasing

soft drinks

Male 90 210

Female 210 490

Example of Independent Random Variables Let random variable M correspond to

whether a customer is male or not and random variable S correspond to whether a customer purchases soft drinks or not.

Then, M and S are independent, since for all possible combinations of the values of M and S, we have

Prob(M=i,S=j)=Prob(M=i)Prob(S=j).

Another Example of Joint Distribution

Object X Y Class Object X Y Class

1 7.1 9.1 1 11 10.9 8.8 2

2 6.7 10.2 1 12 10.8 10.3 2

3 7.5 10.6 1 13 11.1 11 2

4 7.6 8.8 1 14 12.3 9.1 2

5 8.1 10.3 1 15 12.1 9.7 2

6 8.0 11.0 1 16 12 10.9 2

7 8.6 8.9 1 17 13.1 8.9 2

8 8.7 9.8 1 18 12.8 10.1 2

9 9.2 11.2 1 19 13.2 11.3 2

10 6.5 10.1 1 20 13.7 9.9 2

Average 7.8 10.0 - Average 12.2 10.0 -

Joint p.m.f. of X, Y, and C

1

11

1

1

1

1

1

1

1

2

2

2

2

2

2

2

2

2

2

8

9

10

11

12

6 8 10 12 14

Joint p.m.f. of X and C

11 11 11 11 11

22 2 222 22 2 2

0

1

2

6 8 10 12 14

Joint p.m.f. of Y and C

1 1 11 1 11 1 11

2 2 22 2 22 2 22

0

1

2

6 8 10 12 14

Joint Distribution Function

Let X and Y be two random variables. The joint distribution function is defined as follows:

FXY(x,y)=Prob( X≤x, Y≤y). Note that this definition applies to both

discrete and continuous random variables.

Joint Probability Density Function

Assume that X and Y be two continuous random variables defined on the same space S. The joint probability density function of X and Y is defined as follows:

X and Y are said to be independent if and only if

yx

yxFyxf xy

xy

,

,

., yfxfyxf YXXY

In some text books, it is defined that two random variables are independent, if and only if

We have ., yFxFyxF YXXY

.

,,

yfxf

yx

yFxF

yx

yxFyxf

YX

YXXYXY

The marginal p.d.f of X is

and the marginal p.d.f of Y is

dyyxfxf XYX ,

dxyxfyf XYY ,

Jointly Independent and Pairwise Independent

Note that, even we have PX,Y (x,y) = PX (x)PY (y)

PY,Z (y,z) = PY (y)PZ(z)

PX,Z (x,z) = PX (x)PZ (z) Then, it is not necessary true that PX,Y,Z (x,y,z) = PX (x)PY (y) PZ (z)

An Example of Pairwise Independence

Let X and Y are two random variables that correspond to tossing a unbiased coin two times. Let Z = X Y.Then

Prob(Z=0) = Prob(X=0,Y=0) + Prob(X=1,Y=1) = ½

Prob(X=0,Z=0) = Prob(X=0,Y=0) = ¼ = Prob(X=0)Prob(Z=0).

Therefore, X, Y and Z are pairwise independent.However, Prob(X=0,Y=0,Z=1) = 0 and Prob(X=0)Prob(Y=0)Prob(Z=1)= 1/8

Hence, X, Y and Z are not jointly independent.

On the other hand, jointly independent implies pairwise independent. For example,

.

,,, ,,,

yPxP

zPyPxP

zPyPxP

zyxPyxP

YX

zZYX

zZYX

zZYXYX

Addition of Two Random Variables Let X and Y be two random variables. Then,

E[X+Y]=E[X]+E[Y]. Note that the above equation holds even if X

and Y are dependent. Proof of the discrete case :

][][)()(

),(),(

),(),(

))(,(][

YEXEyyPxxP

yxPyyxPx

yxPyyxPx

yxyxPYXE

yY

xX

y xXY

x yXY

x yXY

x yXY

x yXY

On the other hand,

#

2222

2222

222

22

2

])[][][(2][][

)][(2)][()][(

2][2][][

)(2)(])[(

)])((2)()[(

]))()[((

][

YEXEXYEYVarXVar

XYEYEXE

XYEYEXE

YXE

YXYXE

YXE

YXVar

yxyx

yxyx

yxyx

yxyx

yx

Note that if X and Y are independent, then

Therefore, if X and Y are independent, then Var[X+Y]=Var[X]+Var[Y].

][][

)()(

)()(

),(][

YEXE

yyPxxP

yPxxyP

yxxyPXYE

yY

xX

x yYX

x yXY

Covariance

Let X and Y be two random variables. Then, E[(X-µX)(Y- µY)] is called the covariance of X and Y, and is denoted by σXY, where µX and µY are the means of X and Y, respectively.

Covariance

E[(X-µX)(Y- µY)]

= E[XY- µYX- µXY+ µXµY]

= E[XY]- µYE[X]- µXE[Y]+E[µXµY]

= E[XY]- µXµY

Therefore, if X and Y are independent, then Cov[X,Y]=0.

Examples of Correlated Random Variables Assume that a supermarket collected the


Purchasing

Wine

Not Purchasing

Wine

Male 45 255

Female 70 630

Purchasing

Juice

Not Purchasing

Juice

Male 60 240

Female 210 490

Examples of Correlated Random Variables

Let random variable M correspond to whether a customer is male, random variable W correspond to whether a customer purchases wine, random variable J correspond to whether a customer purchases juice.

The joint p.m.f of M and W is

Cov(M,W)= E[MW]-E[M]E[W]= 0.045 – 0.3*0.115 = 0.0105 >0M and W are positively correlated.

W

M

PMW (1,1) = 0.045

PMW (1,0) = 0.255

PMW (0,1) = 0.07

PMW (0,0) = 0.63

The joint p.m.f of M and J is

Cov(M,J)= E[MJ]-E[M]E[J]= 0.06 – 0.3*0.27 = -0.021 < 0M and J are negatively correlated.

W

M

PMW (1,1) = 0.06

PMW (1,0) = 0.24

PMW (0,1) = 0.21

PMW (0,0) = 0.49

Covariance of Independent Random Variables Assume that the supermarket also

collected the following statistics of customers’ purchasing behavior:

Purchasing

soft drinks

Not purchasing

soft drinks

Male 90 210

Female 210 490

The joint p.m.f of M and S is

Cov(M,S)= E[MS]-E[M]E[S]= 0.09 – 0.3*0.3 = 0,due to the fact that M and S are independent.

S

M

PMWS(1,1) = 0.09

PMWS(1,0) = 0.21

PMS (0,1) = 0.21

PMS (0,0) = 0.49

Correlation Coefficient

The correlation coefficient of two random variables X and Y is defined as follows:

YX

YX

),cov(

Bounds of a Correlation Coefficient

.11 Therefore,

. allfor 0 square,a

of valueexpected theis Since

).1()(

have We

.2

))()(()(Let

22

222

2

RbK(b)

K(b)

K

bb

uXbuYEbK

YX

Y

XYXY

XY

Implication of the Value of the Correlation Coefficient Assume that the supermarket collected

the following statistics of customers’ purchasing behavior:

Purchasing

cosmetics

Not purchasing

cosmetics

Male 10 290

Female 260 440

Implication of the Value of the Correlation Coefficient

Let random variable M correspond to whether a customer is male, random variable C correspond to whether a customer purchases cosmetics.

Then, the correlation coefficient of M and C is

-0.349.

Implication of the Value of the Correlation Coefficient On the other hand, we also have the

following dataset:

The correlation coefficient of M and J is -0.103.

Purchasing

juice

Not purchasing

juice

Male 60 240

Female 210 490

Another Example of Correlation Coefficient

Object X Y Class Object X Y Class

1 7.1 9.1 1 11 10.9 8.8 2

2 6.7 10.2 1 12 10.8 10.3 2

3 7.5 10.6 1 13 11.1 11 2

4 7.6 8.8 1 14 12.3 9.1 2

5 8.1 10.3 1 15 12.1 9.7 2

6 8.0 11.0 1 16 12 10.9 2

7 8.6 8.9 1 17 13.1 8.9 2

8 8.7 9.8 1 18 12.8 10.1 2

9 9.2 11.2 1 19 13.2 11.3 2

10 6.5 10.1 1 20 13.7 9.9 2

Average 7.8 10.0 - Average 12.2 10.0 -

Joint p.m.f. of X, Y, and C

1

11

1

1

1

1

1

1

1

2

2

2

2

2

2

2

2

2

2

8

9

10

11

12

6 8 10 12 14

Joint p.m.f. of X and C

11 11 11 11 11

22 2 222 22 2 2

0

1

2

6 8 10 12 14

Joint p.m.f. of Y and C

1 1 11 1 11 1 11

2 2 22 2 22 2 22

0

1

2

6 8 10 12 14

Another Example of Correlation Coefficients

.925.05.0379.2

5.1101.16][][][

CX

CEXEXCE

The correlation coefficient of X and C is

On the other hand, the covariance of Y and C is

E[YC]-E[Y}E[C] = 15-10×1.5 =0

and therefore the correlation coefficient of Y and C is 0.

With respect to data analysis, random variable X provides valuable information about the class of an object.

On the other hand, random variable Y essentially provides no information about the class of an object.

Example of Uncorrelated Random Variables Assume X and Y have the following

joint p.m.fPXY(0,1)= PXY(1,0)= PXY(2,1)= 1/3

We have the following marginal p.m.f.s

xXYY

x xXYYXYX

yXYX

yXYX

xPP

xPPyPP

yPPyPP

3/2)1,(1

3/1)0,(0 ; 3/1),2(2

3/1),1(1 ; 3/1),0(0

Example of Uncorrelated Random Variables

Since PXY(0,1) = 1/3PX(0) x PY(1) = 1/3 x 2/3 = 2/9,X and Y are not independent.

However,Cov (X, Y) = E[XY] – E[X]E[Y] = [2/9 x 1 + 2/9 x 2] – [1 x 2/3] = 0.

Therefore, independence implies uncorrelated, but the inverse is not true.

Discrete Distributions

Documents

Transcript of Discrete Distributions