Random Variables and
Summary Measures
Istanbul Bilgi University
FEC 512 Financial Econometrics-I
Asst. Prof. Dr. Orhan Erdem
FEC 512 Probability Distributions Lecture 2-2
Introduction to Probability Distributions
� Random Variable
� Represents a numerical value from a
random event
Random
Variables
Discrete
Random Variable
Continuous
Random Variable
FEC 512 Probability Distributions Lecture 2-3
Definitions
� The r.v. is discrete if it takes countable
number of values. The discrete r.v. X has
probability density function (pdf) f:R→[0,1]
fiven by f(x)=P(X=x)
� The r.v. is continuous if its takes
uncountable number of values.
FEC 512 Probability Distributions Lecture 2-4
Examples
� Stock prices are discrete random variables,
because they can only take on certain values,
such as 10.00TL, 10.01TL and 10.02TL and
not 10.005TL, since stocks have a minimum
tick size of 0.01TL.
� By way of contrast, stock returns are
continuous not discrete random variables,
since a stock's return could be any number.
FEC 512 Probability Distributions Lecture 2-5
Dicrete Random Variables: Examples
� Roll a die twice: Let x be the number of
times 4 comes up (then x could be 0, 1, or 2
times)
� Toss a coin 5 times.
Let x be the number of heads
(then x = 0, 1, 2, 3, 4, or 5)
FEC 512 Probability Distributions Lecture 2-6
x Value Probability
0 1/4 = .25
1 2/4 = .50
2 1/4 = .25
Experiment: Toss 2 Coins. Let x = # heads.
T
T
Discrete Probability Distribution
4 possible outcomes
T
T
H
H
H H
Probability Distribution
0 1 2 x
.50
.25
Probability
FEC 512 Probability Distributions Lecture 2-7
� 0 ≤ P(xi) ≤ 1 for each xi
� Σ P(xi) = 1
Discrete Probability Distribution
Function (P.d.f.)
FEC 512 Probability Distributions Lecture 2-8
Cumulative Distribution Function(c.d.f.)
� Cumulative distribution function of X is
FX(x)=P(X≤x)
� If X has a pdf then
� Example: Draw the c.d.f of the prev. example
∑=
−∞=
=xu
u
uX fxF )(
FEC 512 Probability Distributions Lecture 2-9
Summary Measures: Location
� Expected Value of a discrete distribution(Weighted Average)
E(x) = Σxi P(xi)
� Example: Toss 2 coins,
x = # of heads,
compute expected value of x:
E(x) = (0 x .25) + (1 x .50) + (2 x .25)=1.0
x P(x)
0 .25
1 .50
2 .25
FEC 512 Probability Distributions Lecture 2-10
The Allais Example-1
� 1 Lottery
� 2.Lottery
� Which one do you prefer?
� It is common for ind. to express 1.Lottery is better than2.Lottery
500,000Outcome
1Probability
0500,0002,500,000Outcome
0.010.890.10Probability
FEC 512 Probability Distributions Lecture 2-11
The Allais Example-2
� 1 Lottery
� 2.Lottery
� Which one do you prefer? It is common for ind. to express
2..Lottery is better than 1.Lottery
500,000
0,11
02,500,00Outcome
0.890Probability
500,000
0
02,500,00Outcome
0.900,10Probability
FEC 512 Probability Distributions Lecture 2-12
� Standard Deviation of a discrete distribution
where:
E(x) = Expected value of the random variable
P(x) = Probability of the random variable having
the value of x
Summary Measures: Dispersion
2
x i i
1
σ {x E(x)} P(x )n
i=
= −∑
FEC 512 Probability Distributions Lecture 2-13
� Example: Toss 2 coins, x = # heads, compute standard deviation (recall E(x) = 1)
Summary Measures: Dispersion
2
x i i
1
σ {x E(x)} P(x )n
i=
= −∑.707.50(.25)1)(2(.50)1)(1(.25)1)(0σ 222
x ==−+−+−=
Possible number of heads
= 0, 1, or 2
FEC 512 Probability Distributions Lecture 2-14
Example: Random Walk
� Assume that at each time step the price can
either increase or decrease by a fixed
amount ∆>0. Suppose that
P1: is the probability of an increase (0< P1 <1)
P2: is the probability of a decrease. (0< P2 <1)
� Random Variable:
If X is the change in a single step, then the set of
possible values of X is {x1= ∆, x2=- ∆} and their
probabilities are {P1, P2}
What is the exp. value of a random walk if P1=P2 =0.5?
FEC 512 Probability Distributions Lecture 2-15
Chebyshev’s Inequality
� Let X be a r.v. with expected value µ and finite
variance σ2. Then for any real number m> 0,
2
2
11)(
1)(
mmXP
or
mmXP
−≤≤−
≤≥−
σµ
σµ
FEC 512 Probability Distributions Lecture 2-16
� Expected value of the sum of two discrete random variables:
E(x + y) = E(x) + E(y)
= Σ x P(x) + Σ y P(y)
Two Discrete Random Variables
FEC 512 Probability Distributions Lecture 2-17
Conditional Expectation
� The conditional pdf of Y given X=x written
fYlX(y l x)=P(Y=ylX=x).
� E(Y l X=x) is called conditional expectation of
Y given X, defined as
E(Y l X)=ΣyfYlX(y l x)
� Although conditional expectation sounds like
a number it is actually a r.v.
FEC 512 Probability Distributions Lecture 2-18
Bivariate Distributions
� Situations where we are interested at the
same time in a pair of r.v. Defined over a joint
sample space.
� If X and Y are disrete r.v., we write the prob
that X will take on the value x and Y will take
on the value y as P(X=x,Y=y), the joint pdf.
� If X and Y are cont r.v. the joint pdf of X,Y is
the function fX,Y(x,y) which display the joint
distribution of X,Y.
FEC 512 Probability Distributions Lecture 2-19
Example
� Determine the value of k for which the
function given by f(x,y)=kxy for x=1,2,3;
y=1,2,3 can serve as a joint pdf.
� Solution: Substituting values of x,y we get
f(1,1)=k; f(1,2)=2k; …f(3,3)=9k
k+2k+3k+2k+4k+6k+3k+6k+9k=1
36k=1 and k=1/36
FEC 512 Probability Distributions Lecture 2-20
Example: Conditional Pdf
Y
X
11/12½5/12
1/36--1/362
7/18-1/62/91
7/121/121/31/60
210
15
1
12/5
36/10)X0YP(
15
8
12/5
9/20)X1YP(
15
6
12/5
6/10)X0YP(
====
====
====
FEC 512 Probability Distributions Lecture 2-21
Continuous Random Variables
� has a probability density function (pdf) fX
such
that
� Examples: Changes in stock prices
( ) ( ) 0
( ) ( ) 1.
( ) , , - ,
( ) ( )
+
-
,
b
a
a f x for a ll x
b f x dx
c For any a b w ith a b
w e have P a X b f x dx
∞
∞
≥
=
∞ < < < +∞
≤ ≤ =
∫
∫
FEC 512 Probability Distributions Lecture 2-22
Cumulative Distribution Function
∫∞−
=
≤=
x
XX
X
duufxF
xXPxF
)()(
then pdf a has X If
)()(
is X of (CDF)function on distributi Cumulative*
FEC 512 Probability Distributions Lecture 2-23
Moments
FEC 512 Probability Distributions Lecture 2-24
Example: Continuous Probability
Distributions
Ex. Suppose that X is a continuous random variable with pdf
Hence the cdf is given by
The graph of F(x) �0
0,2
0,4
0,6
0,8
1
1,2
0 0,2 0,4 0,6 0,8 1 1,2
x
F(x)
( ) 2 , 0 1,
0, .
f x x x
elsewhere
= < <
=
2
0
( ) 0, 0,
2 , 0 1,
1, 1.
x
F x if x
s ds x if x
if x
= ≤
= = < ≤
= >
∫
FEC 512 Probability Distributions Lecture 2-25
Marginal Distributions
Y
X
11/12½5/12
1/36--1/362
7/18-1/62/91
7/121/121/31/60
210
36
12)P(Y ;
18
71)P(Y ;
12
70)P(Y
:Y ofon Distributi Marginal
======
FEC 512 Probability Distributions Lecture 2-26
Conditional Distribution and Expectation
i. Discrete Case
here. fixed is Yhat Remember t
Y.X, of pdf lconditiona theis )(
)()(
Similarly,)(
)()(seen have weBefore
yYP
yYxXPyYxXP
BP
BAPBAP
=
=====
∩=
FEC 512 Probability Distributions Lecture 2-27
Conditional Distribution and Expectation
ii. Continuous Case
( )
( )
( ) ∫∞
∞−
=
=
dyxyyfXYE
xf
yxfxyf
xyf
XY
X
XY
XY
)(
.)(
),(
bygiven is aswritten pdf, lconditiona The
FEC 512 Probability Distributions Lecture 2-28
Martingale
ttt PIPE =+ ][
if Iset info w.r.t martingale a called is P r.v. of sequence a
,I t,at time availableset n informatioan Given
1
tt
t
Some Common Properties
FEC 512 Probability Distributions Lecture 2-30
Skewness
� The skewness of a r.v. measures the symmetry of a
dist. About its mean value.
{ }
continuous is X if
))((
discrete. is X if
)())((
)]([)(
3
3
3
1
3
3
3
x
x
x
n
i
ii
x
fXEX
XPXEX
XEXEXSkew
σ
σ
σ
∫
∑
∞
∞−
=
−
=
−=
−=
FEC 512 Probability Distributions Lecture 2-31
Kurtosis
� The kurtosis of a r.v. measures the thickness in the
tails of a distribution.
{ }
continuous is X if
))((
discrete. is X if
)())((
)]([)(
4
4
4
1
4
4
4
x
x
x
n
i
ii
x
fXEX
XPXEX
XEXEXKurt
σ
σ
σ
∫
∑
∞
∞−
=
−
=
−=
−=
FEC 512 Probability Distributions Lecture 2-32
Example
We know that E(X)=µ=1, σ=0.707 from previous
example.
Skew(X)=[(0-1)30.25+(1-1)3 0.5+(2-1)30.25] /(0.707)3=0.
So it is symmetric.
H.W. Find its kurtosis.
0.250.50.25P(x)
210x
FEC 512 Probability Distributions Lecture 2-33
Covariance
� Covariance between two r.v.
{ }{ }[ ])()( YEYXEXEXY −−=σ
FEC 512 Probability Distributions Lecture 2-34
Covariance (cont.)
If X,Y are discrete r.v:
σxy = Σ [xi – E(x)][yj – E(y)]P(xiyj)
where:
P(xi ,yj) = joint probability of xi and yj.
FEC 512 Probability Distributions Lecture 2-35
Useful Formulas
FEC 512 Probability Distributions Lecture 2-36
� Covariance between two discrete random
variables:
σxy > 0 x and y tend to move in the same
direction
σxy < 0 x and y tend to move in opposite
directions
σxy = 0 x and y do not move closely together
Interpreting Covariance
FEC 512 Probability Distributions Lecture 2-37
Correlation Coefficient
� The Correlation Coefficient shows the strength of the linear association between two variables
where:
ρ = correlation coefficient (“rho”)
σxy = covariance between x and y
σx = standard deviation of variable x
σy = standard deviation of variable y
yx
yx
σσ
σρ =
FEC 512 Probability Distributions Lecture 2-38
� The Correlation Coefficient always falls between -1
and +1
ρ = 0 x and y are not linearly related.
The farther ρ is from 0, the stronger the linear relationship:
ρ = +1 x and y have a perfect positive linear relationship
ρ = -1 x and y have a perfect negative linear relationship
* A strong nonlinear relationship may or or may not imply a
high correlation
Interpreting the Correlation Coefficient
FEC 512 Probability Distributions Lecture 2-39
FEC 512 Probability Distributions Lecture 2-40
Independence
� A r.v. X is independent of Y if knowledge
about Y does not influence the likelihood that
X=x for all possible values of x. and y.
� (Similarly for Y)
� Holds for both type of r.v.
FEC 512 Probability Distributions Lecture 2-41
Independence
not true. is converse The
edness.uncorrelatE(X)E(Y)=E(XY)ceindependen Thus
eduncorrelat are Y and X then E(X)E(Y),=E(XY) If
E(X)E(Y)=E(XY)t then independen are Y and X If
R.y x,allfor
disc.r.v.for y)x)P(YP(Xy)andYx P(Xor
r.v.cont for (y)(x)ff=y)(x,f
ifonly and ift independen are Y and X r.v. The
YXYX,
⇒⇒
∈
=====
FEC 512 Probability Distributions Lecture 2-42
Linear Functions of a Random Variable
� Let X be a r.v. Either discrete or cont. E(X)=µ,
Var(X)=σ2.Define a new r.v. Y as
Y=aX+b.
� Then E(Y)=aE(X)+b=a µ+b
Var(Y)=a2 σ2
FEC 512 Probability Distributions Lecture 2-43
Linear Combinations of Two Random
Variables
� Let X~(µX,σX2 ) and Y~(µY,σY
2 ) and
σXY=cov(X,Y).
If Z=aX+bY where a,b are constants, then
Z~(µZ,σZ2 ) where
µZ=a µX +b µY
σZ2=a2 σX
2 +b2 σY2 +2abσXY=a2 σX
2 +b2 σY2
+2abσX σYρ
FEC 512 Probability Distributions Lecture 2-44
Linear Combinations of N Random
Variables
FEC 512 Probability Distributions Lecture 2-45
Diversification
� As long as security returns are not positively
correlated, diversification benefits are possible. The
smaller the correlation between security returns, the
greater the cost of not diversifying.
� σZ2=a2 σX
2 +b2 σY2 +2abσX σYρ
10.50-0.5-1
3
2.5
2
1.5
1
0.5
0
Correlation
Sigma(Z)
Correlation
Sigma(Z)