ENSC327

Communications Systems

17: Random Variables and Expectation

1

Jie Liang

School of Engineering Science

Simon Fraser University

Introduction to Random Variables

� Definition:� A random variable is a mathematical function that maps the outcomes of random experiments to numbers.

� Caution:� Random variable is actually a function, not a variable.

2

Random variable is actually a function, not a variable.

� Notation:� Random variables: Capital letters (e.g., X, Y, Z)

� Values of random variables: lowercase letters, x, y, z…

� Examples:� A student’s grade in the midterm exam

� Price of Google’s stock

� Average temperature of each day

� Number of heads when tossing 10 fair coins

Random Variables (r.v.’s)

�Map the abstract sample spaces to the real line

� events (collections of outcomes) become intervals

1z ( )X g

3

W

2z

1( )X z

2( )X z

( )X g

� Enables a “simple” probability function

� instead of a set-oriented probability measure

Random Variables (r.v.’s)� The probability functions applies to the real-valued random

variable in the same manner as it applies to the events.

� Probability analysis can be developed in terms of real-valued

quantities regardless of the form or shape of the underlying

events of the random experiment.

4

Cumulative Distribution Function (cdf)

�The cdf completely describes the probability

distribution of a real-valued random variable

Definition:

5

� Note:

The book refers to CDF as the “probability

distribution function”, which can be easily confused

with the “probability density function” f(x).

Therefore CDF is the more commonly used name

for F(x), and will be used in this course.

Properties of CDF

� 3. is continuous from the right( )X

F x

1.

2.

6

� 3. is continuous from the right( )X

F x

Ziemer

Example

� If , where u(x) is the unit step function, find P(X > 5), P(X>5|X<7).

� Solution:

)()1()( xuexFx

X

−

−=

)(xFX

7

Probability Density Functions (pdf)

� Applicable to continuous random variables.

� Definition:

( )( )

X

X

dF xf x

dx=

8

�Properties:

Example of pdf

� The final angle (modulo ) when a wheel

is spin (with respect to the vertical line)

� Assumption: The pointer is equally

possible to stop at any position

2π θ

9

possible to stop at any position

Probability Mass Function (pmf) for

Discrete Random Variable

� Suppose that X is a discrete random variable, taking

values on some countable sample space S. Then the

probability mass function for X is given by

∈=

=, ],[

)(SxxXP

xf

10

∈=

=otherwise. ,0

, ],[)(

SxxXPxf

X

)(xfX� Note that is defined for all real numbers,

including all values that X could never take; indeed,

it assigns such values a probability of zero.

Probability Mass Function (pmf) for

Discrete Random Variable

� Example: Let X be the random variable representing

the outcome of a fair coin-tossing experiment.

� Let X=0 if the result is tail and 1 if its head.

� The PMF is:

11

� The PMF is:

Several random variables

� Some random experiments must be characterized by

two or more random variables

� Joint cdf of X and Y:

[ ]yYxXPyxFYX

≤≤= ,),(,

X

Y

12

�Properties:

�Joint pdf of X and Y:

[ ]yYxXPyxFYX

≤≤= ,),(,

X

yx

yxFyxf

YX

YX

∂∂

∂=

),(),(

,

2

,

X

Y

Example

� How to write

� Solution:

[ ] ?F of in terms ,XY2121

yYyxXxP ≤<≤<

13

Example

14

Marginal pdf or cdf

[ ] ),(,)( ∞=∞≤≤= xFYxXPxFXYX

[ ] ),(,)( yFyYXPyF ∞=≤∞≤=

15

[ ] ),(,)( yFyYXPyFXYY

∞=≤∞≤=

Conditional pdf

=)|(| xyfXY

=)|( yxf

16

=)|(| yxfYX

=)|(| yxfYX

Baye’s rule:

Independent Random Variables

�Two random variables are independent if and

only if

17

Or equivalently:

Independent Random Variables

Since

So if X and Y independent:

18

So if X and Y independent:

Statistical Averages� Statistical averages can reveal many useful properties

of random variables

� Especially when cdf and pdf are unknown.

� The mean (expectation) of discrete r.v. X:

19

�When P[X=x] are unknown, use the average

of N observations x1, x2, …, xN:

Statistical Averages

� Expectation of continuous random variable X:

20

� Expectation of linear combination of r.v.'s:

� Expectation of function of random variable:

� If Y = g(X):

Variance

�Measure of the spread around the mean value.

[ ]{ }22 XX

XE µσ −=

� Property: 222 )(XX

XE µσ −=

deviation. standard :Xσ

21

� Property: )(XX

XE µσ −=

Variance

� Proof for continuous r.v:

22

Covariance

�Covariance of two r.v.'s X and Y:

� X and Y are said to be uncorrelated if Cov(X, Y) = 0, or

E(XY)= YXµµ

23

� If X and Y are independent, i.e., f(x,y) = f(x)f(y), then

E(XY)= YX

Covariance

� So Independence implies uncorrelated:

independence is a stronger condition.

24

�However, the converse is not always true

� If X and Y are uncorrelated, they are not

necessarily independent.

� But if X, Y are Gaussian (studied in next lecture),

independence and uncorrelated will be equivalent.

Correlation Coefficient

�The normalized covariance is called

correlation coefficient between two random

variables:

.),( YXCov

σσρ =

25

YXσσ

� It can be shown that

� If X and Y are uncorrelated �

[ ].1 ,1−∈ρ

.0=ρ

Correlation Coefficient

� If Y = aX + b, then

� Proof:

<−

>=

0.a if ,1

0.a if ,1ρ

Let X has mean and variance:

Then the mean and variance of Y are:

. and2

XXσµ

{ }YX

XYEYXCov µµ−=),(

26

Then the mean and variance of Y are:

Correlation Coefficient { }YX

XYEYXCov µµ−=),(

27

Additional Properties of Variance

�If Xi are uncorrelated:

.)(1

2

1

∑∑==

=

�

i

ii

�

i

iiXVaraXaVar

28