DC-02-Probability and Stochastic Processes

Probability and Stochastic ProcessesProbability and Stochastic Processes

Wireless Information Transmission System Lab.Wireless Information Transmission System Lab.Institute of Communications EngineeringInstitute of Communications Engineeringg gg gNational Sun National Sun YatYat--sensen UniversityUniversity

Table of ContentsTable of Contents

◊ Probability◊ Random Variables, Probability Distributions, and Probability Densities, y , y◊ Statistical Averages of Random Variables◊ Some Useful Probability Distributions

U B d th T il P b bilit◊ Upper Bounds on the Tail Probability◊ Sums of Random Variables and the Central Limit Theorem

◊ Stochastic Processes◊ Statistical Averages

P D i S◊ Power Density Spectrum◊ Response of a Linear Time-Invariant System to a Random Input Signal◊ Discrete-Time Stochastic Signals and Systemsg y◊ Cyclostationary Processes

2

ProbabilityProbability

◊ Sample space or certain event of a die experiment:

◊ The six outcomes are the sample points of the 6,5,4,3,2,1S

◊ The six outcomes are the sample points of the experiment.

◊ An event is a subset of S and may consist of any◊ An event is a subset of S, and may consist of any number of sample points. For example:

42A◊ The complement of the event A, denoted by ,

4,2AA

consists of all the sample points in S that are not in A: 6531A

3

6,5,3,1A


◊ Two events are said to be mutually exclusive if they h l i i h i if hhave no sample points in common – that is, if the occurrence of one event excludes the occurrence of the

h F lother. For example: 6,3,1 ;4,2 BA

◊ The union (sum) of two events in an event that consistsevents. exclusivemutually are and AA

◊ The union (sum) of two events in an event that consists of all the sample points in the two events. For example:

C 3,2,1 CBD

C 6,3,2,1

3,2,1

4

SAA


◊ The intersection of two events is an event that consists◊ The intersection of two events is an event that consists of the points that are common to the two events. For example:example: 3,1 CBE

◊ When the events are mutually exclusive, the intersection is the null event, denoted as φ. For

lexample:AA

5


◊ Associated with each event A contained in S is its probability P(A).

◊ Three postulations:p◊ P(A)0.◊ The probability of the sample space is P(S)=1. p y p p ( )◊ Suppose that Ai , i =1, 2, …, are a (possibly infinite) number

of events in the sample space S such that

Then the probability of the union of these mutually ,...2,1 ; jiAA ji

p y yexclusive events satisfies the condition:

ii APAP )(6

i

ii

i )(


◊ Joint events and joint probabilities (two experiments)◊ If one experiment has the possible outcomes Ai , i =1,2,…,n, and

the second experiment has the possible outcomes Bj , j =1,2,…,m, then the combined experiment has the possible joint outcomesthen the combined experiment has the possible joint outcomes(Ai ,Bj), i =1,2,…,n, j =1,2,…,m.

◊ Associated with each joint outcome (Ai ,Bj) is the joint probabilityjP (Ai ,Bj) which satisfies the condition:

1),(0 ji BAP◊ Assuming that the outcomes Bj , j =1,2,…,m, are mutually

exclusive, it follows that:

m

iji APBAP )(),(

◊ If all the outcomes of the two experiments are mutually exclusive, then:

j 1

n m n

7

1 1 1

, 1i j ii j i

P A B P A


◊ Conditional probabilitiesp◊ The conditional probability of the event A given the

occurrence of the event B is defined as:

id d ( ))(

),()|(BP

BAPBAP

provided P(B)>0.◊ )()|()()|(),( APABPBPBAPBAP

iTh tfb bilitthdi t ti)( BABAP ◊

◊ If two events and are mutually exclusiveA B A B . and of occurrence ussimultaneo thedenotes ),(

is,That .ofy probabilittheasdinterpreteis ),(BABAP

BABAP

◊

◊

If two events and are mutually exclusive, , then ( | ) 0.

A B A BP A B

1)|(andhaveweofsubsetaisIf BAPBBAAB

8

◊ .1)|(andhavewe,ofsubset a is If BAPBBAAB


◊ Bayes’ theorem:◊ If , 1, 2,..., , are mutually exclusive events such thati

n

A i n

A S

i 1

and is an arbitrary event with nonzero probability, then

iA S

B

( , ) ( | ) ( )

ii

P A B PP A BP B

( | ) ( )

( | ) ( )

i in

j j

B A P A

P B A P A1

( | ) ( )j jj

P B A P A

1 1

, |n n

j j jj j

P B P B A P B A P A

◊ P(Ai) represents their a priori probabilities and P(Ai|B) is the a posteriori probability of Ai conditioned on having observed

9

p p y i gthe received signal B.


◊ Statistical independence◊

).()|( then , ofoccurrence on the dependnot does of occurrence theIf

APBAPBA

◊

◊ When the events A and B satisfy the relation )()()()|(),( BPAPBPBAPBAP

P(A,B)=P(A)P(B), they are said to be statistically independent.◊ Three statistically independent events A1, A2, and A3 must

satisfy the following conditions:)()(),( 2121 APAPAAP

)()(),()()(),(

3232

3131

APAPAAPAPAPAAP

10

)()()(),,( 321321 APAPAPAAAP

Random Random Variables, Probability Distributions, and Variables, Probability Distributions, and Probability DensitiesProbability Densities

◊ Given an experiment having a sample space andS

robab l ty Dens t esrobab l ty Dens t es

elements , we define a funciton ( ) whose domainis and whose range is a set of numbers on the real line.

s S X sS

◊ The function X(s) is called a random variable.◊ Example 1: If we flip a coin, the possible outcomes are head (H) and tail (T),

so S contains two points labeled H and T Suppose we define a function X(s)so S contains two points labeled H and T. Suppose we define a function X(s) such that:

Th h d th t ibl t f th i fli i

T)(s 1-H)(s 1

)(sX

Thus we have mapped the two possible outcomes of the coin-flipping experiment into the two points ( +1,-1) on the real line.

◊ Example 2: Tossing a die with possible outcomes S={1,2,3,4,5,6}. A random variable defined on this sample space may be X(s)=s, in which case the outcomes of the experiment are mapped into the integers 1,…,6, or, perhaps, X(s)=s2, in which case the possible outcomes are mapped into the integers

11

( ) , p pp g{1,4,9,16,25,36}.


◊ Give a random variable X, let us consider the event {X≤x} where


{ }x is any real number in the interval (-∞,∞). We write the probability of this event as P(X ≤x) and denote it simply by F(x), ii.e.,

h f i ( ) i ll d h b b l d b f

( ) ( ), F x P X x - x

◊ The function F(x) is called the probability distribution functionof the random variable X.It i l ll d th l ti di t ib ti f ti (CDF)◊ It is also called the cumulative distribution function (CDF).

◊ 1)(0 xF◊ .1)( and 0)( FF

12


◊ Examples of the cumulative distribution functions of


ptwo discrete random variables.

13


◊ An example of the cumulative distribution function of a


pcontinuous random variable.

14


◊ An example of the cumulative distribution function of a


prandom variable of a mixed type.

15


◊ The derivative of the CDF F(x), denoted as p(x), is


( ), p( ),called the probability density function (PDF) of the random variable X.

d )( x

dxxdFxp ,)()(

x

xduupxF ,)()(

◊ When the random variable is discrete or of a mixed type, the PDF contains impulses at the points of discontinuity of F(x):

n

ii xxxXPxp )()()(

16

i 1


Determining the probability that a random variable X


1 2 2 1

g p yfalls in an interval , , where .x x x x

2 1 1 2( ) ( ) ( )P X x P X x P x X x

2 1 1 2

1 2 2 1

( ) ( ) ( ) ( ) ( ) ( )

F x F x P x X xP x X x F x F x

1 2 2 1( ) ( ) ( )

p 2

1( )x

xx dx 1 2

1 2

The probability of the event is simplythe area under the PDF in the range .

x X xx X x

17

1 2the area under the PDF in the range .x X x


◊ Multiple random variables, joint probability distributions,


and joint probability densities: (two random variables)),(),(),( :CDFJoint 2

x

1212211211 2

udduuupxXxXPxxFx

),(),( :PDFJoint 2121

2

21

- -

xxFxx

xxp

)(),( )(),( 12212121

21

xpdxxxpxpdxxxp

xx

PDFs. called are variables theofoneover gintegratin from obtained )( and )( PDFs The 21 xpxp

marginal

0:thatNote

1,),( 2- - 121

xFxFF

Fxddxxxp

18

.0,,, :thatNote 12 xFxFF


◊ Multiple random variables, joint probability distributions, and j i b bili d i i ( l idi i l d i bl )


joint probability densities: (multidimensional random variables). variablesrandom are 21 that Suppose i ,...,n,,, iX

)(

),...,,(),...,,( CDFJoint x

221121

1 2

x x

nnn

duudduuuup

xXxXxXPxxxFn

),...,,(),...,,( PDFJoint

...),...,,(...

2121

- 2- 121

n

n

n

nn

xxxFxxxp

duudduuuup

),...,,(),...,,(

), ,,(...

), ,,(

413221

2121

21

nn

nn

n

xxxpdxdxxxxp

xxxp

0

).,...,,,(,...,,,,

), ,,(), ,,(

54141

413221

nn

nn

xxxFxxxxFxxxF

pp

19

.0,...,,,, 41 nxxxF

Statistical Statistical Averages of Random VariablesAverages of Random Variables

◊ The mean or expected value of X, which characterized◊ The mean or expected value of X, which characterized by its PDF p(x), is defined as:

dxxxpmXE )()(This is the first moment of random variable X.

dxxxpmXE x )()(

◊ The n-th moment is defined as:dxxpxXE nn )()(

◊ Define Y=g(X), the expected value of Y is:

dxxpxgXgEYE )()()()(

20


◊ The n-th central moment of the random variable X is:

dxxpmxmXEYE n

xn

x )()()(

◊ When n=2, the central moment is called the varianceof the random variable and denoted as :2

x22 )()( xx dxxpmx

◊ In the case of two random variables, X1 and X2, with 22222 )()()( xx mXEXEXE

◊ In the case of two random variables, X1 and X2, with joint PDF p(x1,x2), we define the joint moment as:

21

21212121 ),()( dxdxxxpxxXXE nknk


◊ The joint central moment is defined as:◊ The joint central moment is defined as:

2211 )()( mXmXE nk

21212211 ),()()( dxdxxxpmxmx nk

◊ If k=n=1, the joint moment and joint central momentare called the correlation and the covariance of the random variables X1 and X2, respectively.

◊ The correlation between Xi and Xj is given by the joint◊ The correlation between Xi and Xj is given by the joint moment:

jijijiji dxdxxxpxxXXE ),()(

22

jijijiji p ),()(


◊ The covariance between Xi and Xj is given by the joint central jmoment:

jjiiij mXmXE

][

jiijjii

dxdxxxpmmmxmxxx

dxdxxxpmxmx j

)(

),(

jijijiji

jiijijiijji

mmdxdxxxpxx

dxdxxxpmmmxmxxx j

),(

),(

jijijijijiji

jijijiji

mmmmmmdxdxxxpxx

),(

◊ The n×n matrix with elements μij is called the covariance matrixjiji mmXXE )(

23

μijof the random variables, Xi, i=1,2, …, n.


◊ Two random variables are said to be uncorrelated if E(XiXj)=E(Xi)E(Xj)=mimj.

◊ Uncorrelated → Covariance μij = 0.◊ Uncorrelated Covariance μij 0.◊ If Xi and Xj are statistically independent, they are

uncorrelateduncorrelated.◊ If Xi and Xj are uncorrelated, they are not necessary

t ti ti ll i d d tlstatistically independently.◊ Two random variables are said to be orthogonal if

E(XiXj)=0.◊ Two random variables are orthogonal if they are uncorrelated

24

and either one or both of them have zero mean.


◊ Characteristic functions◊ The characteristic function of a random variable X is

defined as the statistical average:

dxxpejveE jvxjvX )()()(

◊ Ψ(jv) may be described as the Fourier transform of p(x).◊ The inverse Fourier transform is:

dvejvxp jvx)(21)(

◊ First derivative of the above equation with respect to v:

dxxpxejjvd jvx )()(

25

dxxpxej

dv)(


◊ Characteristic functions (cont.)( )◊ First moment (mean) can be obtained by:

0

)()(

v

x dvjvdjmXE

◊ Since the differentiation process can be repeated, n-th

0v

◊ Since the differentiation process can be repeated, n thmoment can be calculated by:

0

)()()(

v

n

nnn

dvjvdjXE

26


◊ Characteristic functions (cont.)( )◊ Determining the PDF of a sum of statistically independent

random variables: n

ii

jvYY

n

ii XjvEeEjvXY exp)()(

11

nn

n

i

jvxn

i

jvX dxdxdxxxxpeeE ii ...),...,,(... 212111

n

XYnn jvjvxpxpxpxxxp )()( )()...()(),...,,(

t,independenlly statisticaarevariablesrandom theSince

2121

i

iXYnn

X

jjppppi

d)distributey identicall andnt (independe iid are If

)()()()()(), ,,(1

2121

27

nXY jvjv )()(


◊ Characteristic functions (cont.)( )◊ The PDF of Y is determined from the inverse Fourier

transform of ΨY(jv).◊ Since the characteristic function of the sum of n statistically

independent random variables is equal to the product of the characteristic functions of the individual random variables, it follows that, in the transform domain, the PDF of Y is the n-fold convolution of the PDFs of the Xfold convolution of the PDFs of the Xi.

◊ Usually, the n-fold convolution is more difficult to perform than the characteristic function method in determining the PDFthan the characteristic function method in determining the PDF of Y.

28

Some Useful Probability DistributionsSome Useful Probability Distributions

◊ Binomial distribution (discrete):◊

◊ Let pXPXP 110

where the 1 2 are statistically iidn

Y X X i n what is the probability distribution function of Y ?

1

where the , 1, 2,..., are statistically iid,i ii

Y X X i n

!!! 1

knkn

kn

ppkn

kYP knk

)( :is of PDF0

kykYPypYn

k

)()1(

0

kyppkn knk

n

k

29

0 kk

Some Useful Probability DistributionsSome Useful Probability Distributions◊ Binomial distribution:

◊ The CDF of Y is:

knky

ppn

yYPyF

)1(

where [y] denotes the largest integer m such that m≤y.

k

ppk

yYPyF

)1(0

◊ The first two moments of Y are: npYE

)1()1()(

2

222

pnppnpnpYE

◊ The characteristic function is:

njvj )1()(

30

njvpepj )1()(


◊ Uniform Distribution◊ The first two moments of X are:

1

222 1

)(21 baXE

22

222

)(1

)(31)(

b

abbaXE


22 )(12

ba


)()(

beej

jvajvb

31

)()(

abjvj


◊ Gaussian (Normal) Distribution◊ The PDF of a Gaussian or normal distributed random variable

is:22 2/)(1)( mx

h i th d 2 i th i f th d

2/)(

21)(

xmxexp

where mx is the mean and σ2 is the variance of the random variable.

◊ The CDF is: ,

2 2xu m dut dt

◊ The CDF is:

11)(2/2/ 222 mx tx mu dteduexF x

x

2 2

)(11)(112

)(

xx mxerfcmxerf

dteduexF

32

)2

(2

1)2

(22

erfcerf


◊ Gaussian (Normal) Distribution◊ erf( ) and erfc( ) denote the error function and complementary

error function, respectively, and are defined as:

)(12 and 2 22

0xerfdtexerfcdtexerf

x

tx t

◊ erf(-x)=-erf(x), erfc(-x)=2-erfc(x), erf(0)=erfc(∞)=0, anderf(∞)=erfc(0)=1.erf(∞) erfc(0) 1.

◊ For x>mx, the complementary error functions is proportional to the area under the tail of the Gaussian PDF.

33


◊ Gaussian (Normal) Distribution◊ The function that is frequently used for the area under the tail

of the Gaussian PDF is denoted by Q(x) and is defined as:

2 / 21 1( ) 022 2

t

x

xQ x e dt erfc x

22 2x

34


◊ Gaussian (Normal) Distribution◊ The characteristic function of a Gaussian random variable

with mean mx and variance σ2 is:1

Th t l t f G i d i bl

2222 2/12/

21

vjvmmxjvx xx edxeejv

◊ The central moments of a Gaussian random variable are:

)(even )1(31

)(kk

mXEk

k

◊ The ordinary moments may be expressed in terms of the

) (odd 0)(

kmXE kx

◊ The ordinary moments may be expressed in terms of the central moments as:

ik

k kXE

35

ikix

i

k mi

XE

0


◊ Gaussian (Normal) Distribution◊ The sum of n statistically independent Gaussian random

variables is also a Gaussian random variable.

1

n

iiXY

2/

1

2/

1

2222 vjvmn

i

vjvmn

iXY eejvjv yyii

i

and where1

22

1

n

iiy

n

iiy mm

varianceand

mean with ddistribute-Gaussian is Therefore,2

ymY

36

. varianceand y


◊ Chi-square distribution◊ If Y=X2, where X is a Gaussian random variable, Y has a chi-

square distribution. Y is a transformation of X.Th f hi di ib i◊ There are two type of chi-square distribution:◊ Central chi-square distribution: X has zero mean.◊ Non-central chi-square distribution: X has non-zero mean◊ Non central chi square distribution: X has non zero mean.

◊ Assuming X be Gaussian distributed with zero mean and variance σ2, we can apply (2.1-47) to obtain the PDF of Y with a=1 and b=0;

]/[]/[ 11 abyxpabyxp XX

]/[

][]/[][

1

1

1

1

abyxgyp

abyxgyp

yp XXY

37


◊ Central chi-square distributionTh PDF f Y i◊ The PDF of Y is:

0,1)(22/ yeyp y

Y


0 ,2

)( yey

ypY


dueduupyF uyy

YY

22/11)(

◊ The characteristic function of Y is:

u

py YY 00 2)(

2/12 )21(1)(

vj

jvY

38

)21( vj


◊ Chi-square (Gamma) distribution with n degrees of f dfreedom.◊ andt independenlly statistica are ,,...,2,1 , ,2 niXXY i

n

i with variablesrandomGaussian (iid) ddistributey identicall

2

1i

◊ The characteristic function is:.varianceandmean zero 2

1

◊ The inverse transform of this characteristic function yields 2/2 )21(

1)( nY vjjv

ythe PDF:

0 12

1)(2212

2/

y,e ynσ

yp σ-y/n/

nnY

39

22

nσ


◊ Chi-square (Gamma) distribution with n degrees of freedom (cont.).◊ :asdefinedfunction,gamma theis )( p

0integeran!1p)(

0 ,)( 0

1

pp

pdtetp tp

21

23

21

0integer an !1-p)(

pp

◊ When n=2, the distribution yields the exponential distribution.

222

distribution.

40

Some Useful Probability DistributionsSome Useful Probability Distributions◊ Chi-square (Gamma) distribution with n degrees of

freedom (cont )freedom (cont.).◊ The PDF of a chi-square distributed random variable for

several degrees of freedom.g

41


◊ Chi-square (Gamma) distribution with n degrees of freedom (cont.).◊ The first two moments of Y are:

4242

2

2)()(

nnYEnYE

Th CDF f Y i

42 2)( ny


y un ydueuyF 2/12/ 01 2

nn

Y ydueun

yF0 2/

0 ,

212

42


◊ Chi-square (Gamma) distribution with n degrees of freedom (cont.).◊ The integral in CDF of Y can be easily manipulated into the

f f h i l f i hi h i b l dform of the incomplete gamma function, which is tabulated by Pearson (1965).

◊ When n is even the integral can be expressed in closed form◊ When n is even, the integral can be expressed in closed form. Let m=n/2, where m is an integer, we can obtain:

0 ,)2

(!

11)( 2

1

0

2/ 2

yyk

eyF km

k

yY

43


◊ Non-central chi-square distribution◊ If X is Gaussian with mean mx and variance σ2, the random

variable Y=X2 has the PDF:

0 ),cosh(21)( 2

2/)( 22

ymy

ey

yp xmyY

x

◊ The characteristic function corresponding to this PDF is:

2 y

)21/(2/12

22

)21(1)(

vjvjm

Yxe

vjjv

44


◊ Non-central chi-square distribution with n degrees of freedom◊ andt independenlly statistica are ,,...,2,1 , ,2 niXXY i

n

i with variablesrandomGaussian (iid) ddistributey identicall2

1i

◊ The characteristic function is:.toequal varianceidenticaland,,...,2,1 ,mean 2nimi

1

2

exp1)(mjv

jv

n

ii

Y

22/2 21

exp)21(

)(

vjvj

jv nY

45


◊ Non-central chi-square distribution with n degrees of freedom◊ The characteristic function can be inverse Fourier

f d i ld h PDFtransformed to yield the PDF:2 2( 2) / 4 ( ) / 2

/ 2 12 2 2

1( ) ( ) ( ), 02

n s yY n

y sp y e I y ys

2

2 2

2where, is called the non-centrality parameter:

n

ss

s m

1

and is the th-order modified Bessel function

ii

s m

I x

of the

2

first kind, which may be represented by the infinite series:( / 2) ( ) , 0

( )

kxI x xk k

46

0

( ) ,! ( 1)k k k


◊ Non-central chi-square distribution with n degrees of freedom◊ The CDF is:

y

nus

n

Y dusuIesuyF

0 212/2/)(

4/2

22 )(2

1)(22

◊ The first two moments of a non-central chi-square-distributed random variable are:

s2

distributed random variable are:

22)( snYE

2242

2222242

42)(42)(

)(

snsnsnYE

47

42 sny


◊ Non-central chi-square distribution with n degrees of freedom◊ When m=n/2 is an integer, the CDF can be expressed in

terms of the generalized Marcum’s Q function:1

2 2

2 2

1( ) / 2

1

1

, ( )m

x am mb

km

xQ a b x e I ax dxa

b

2 2

2 2

1/ 21

0

/ 2

,ma b

kk

ka b

bQ a b e I aba

b

/ 21

022 2

2

where , , 0

By using and let it is easily shown:

a bk

k

bQ a b e I ab b aa

u sx a

22By using and let , it is easily shown:

x a

( ) 1 ,Y mysF y Q

48


◊ Rayleigh distributiony g◊ Rayleigh distribution is frequently used to model the statistics

of signals transmitted through radio channels such as cellular radio.

◊ Consider a carrier signal s at a frequency ω0 and with an amplitude a:

)exp( 0tjas

◊ The received signal sr is the sum of n waves:

n

ii tjrtjas 00 )(exp)(exp

n

ii

iiir

jajr

tjrtjas 01

0

)exp()exp(where

)(exp)(exp

49

i

ii jajr1

)exp()exp( where


◊ Rayleigh distributiony g◊ sincos)exp( :Define

11

jyxajajrn

iii

n

iii

sin and cos :have We11

ayaxn

iii

n

iii

◊ Because (1) n is usually very large, (2) the individual

sin cos :where 222 ryrxyxr

◊ Because (1) n is usually very large, (2) the individual amplitudes ai are random, and (3) the phases θi have a uniform distribution, it can be assumed that (from the central limit theorem) x and y are both Gaussian variables with means equal to zero and variance:

50

222 yx


◊ Rayleigh distribution◊ Rayleigh distribution◊ Because x and y are independent random variables, the joint

distribution p(x,y) isp( ,y)

2

22

2 2exp

21)()(),(

yxypxpyxp

◊ The distribution p(r,θ) can be written as a function of p(x,y) :

)()( yxpJrp

sincos//

),(),(

rrxrx

J

yxpJrp

2

exp)(

cossin//

rrrp

ryry

51

22exp

2),(

rp


◊ Rayleigh distribution◊ Rayleigh distribution◊ Thus, the Rayleigh distribution has a PDF given by:

22 2/2 r

otherwise 0

0 ),()(22 2/

22

0

rerdrprp

r

R

◊ The probability that the envelope of the received signal does not exceed a specified value R is given by the corresponding

not exceed a specified value R is given by the corresponding cumulative distribution function (CDF):

r

0 ,exp1)(2222 2/

0

2/2 rdueurF r

ru

R

52


◊ Rayleigh distribution

◊ Mean: 2533.12

)(][0

drrrpRErmean

2

◊ Variance:2

0

2222

2)(][][

drrprREREr

22 4292.02

2

medianr1

◊ Median value of r is found by solving: median drrp

021

177.1medianr 2/ kk

◊ Monents of R are:

lik l l

212][ 2/2 kRE kk

53

◊ Most likely value:= max { pR(r) } = σ.


◊ Rayleigh distributiony g

54


◊ Rayleigh distributiony g◊ Probability That Received Signal Doesn’t Exceed A Certain

Level (R) )()( dFr

2

0

)()(

uu

duuprF

r

R

2

022 2

exp

duuu

r

02

2

2exp

u

2

2

2exp1

r

55


◊ Rayleigh distributiony g◊ Mean value:

[ ] ( )meanr E R rp r dr

0

2 2 2

2 2 2exp exp2 2

r r rdr rd

0 0

2 2

2 2

2 2

exp exp2 2r rr dr

0-0=0

2 200

2

p p2 212 exp r dr

20

2 exp22

12 1 2533

dr

56

2 1.25332 2


◊ Rayleigh distribution:y g◊ Mean square value:

22

223

0

22 )(][

rrr

drrprRE

20

2

022 2

exp 2

exp

rdrdrrr

0-0=0

2

2

002

22

2exp2

2exp

drrrrr0-0=0

22

22 2

2exp2

r

57

02


◊ Rayleigh distributiony g◊ Variance:

222 ][][

REREr

22 )2

()2(

22 4292.02

2

58


◊ Rayleigh distributiony g◊ Most likely value

◊ Most Likely Value happens when: dp(r) / dr = 0

021)( 222

rrrrdp

02

exp22

exp)(2422

rdrp

6065021exp

2

rr

r

6065.022

exp)( 22

rr

rrrp

59


◊ Rayleigh distributiony g◊ Characteristic function

2/2

22

jvrr

R dreerjv

sinrcosr

0

2/20

2/2

0 2

2222

rr

R

drvrejdrvre

dreejv

21

21;

21,1 2/222

11

0 20 2

22

vevjvF

j

:functionetrichyupergeomconfluent theis;21,1where

222

11

11

aF

j

,...2,1,0 ,;;

y p g2

11

11

k

kkxkxF

60

,,,,!

;;0

11

k kk


◊ Rayleigh distributiony g◊ Characteristic function (cont.)

1

11

:asexpressed

bemay ;21,1shown thathas(1990)Beaulieu aF

11 !12

;21,1

:asexpressedk

a

kkaeaF

0 !122 k kk

61


◊ Rice distribution◊ When there is a dominant stationary (non-fading) signal

component present, such as a line-of-sight (LOS) propagation path, the small-scale fading envelope distribution is Rice.

esdirect wav wavesscattered

)](exp[)exp(])[()exp()](exp['

00

00

tjrtjjyAxtjAtjrsr

)(

)](exp[)exp(])[( 222

00

yAxr

tjrtjjyAx

sincos

ryrAx

62

sinry


◊ Rice distribution◊ By following similar steps described in Rayleigh distribution,

we obtain:

2 2

02 2 2exp for A 0,r 0 ( ) 2r r r

r r A ArIp r

0 for 0where

(r )

2

0 2 20

1 cos exp2r r

Ar ArI d

is the modified zeroth-order Bessel function.

ixI (x)

63

00

! 2i

iI (x)

i


◊ Rice distribution◊ The Rice distribution is often described in terms of a

parameter K which is defined as the ratio between the d i i i i l d h i f h l i hdeterministic signal power and the variance of the multi-path. It is given by K=A2/(2σ2) or in terms of dB:

2

Th t K i k th Ri f t d l t l

2

2(dB) 10 log [dB]2AK

◊ The parameter K is known as the Rice factor and completely specifies the Rice distribution.

◊ As A0 K ∞ dB and as the dominant path decreases in◊ As A0, K-∞ dB, and as the dominant path decreases in amplitude, the Rice distribution degenerates to a Rayleigh distribution.

64


◊ Rice distribution

65



n

iiir tjAtjas 0

10 )exp()(exp

n

ii

i

tjAtjja 00

1

)exp()()exp(

i

tjAtjjr 00

1

)exp()exp()exp('

tjrtjjyAxtjAtjr

00

00

)(exp)exp()( )exp()(exp'

n

ii jajr

jjjy 00

)exp()exp(' where

)(e p)e p()(

66

i 1


◊ Rice distribution◊ sincos)exp(' :Define

11

jyxajajrn

iii

n

iii

sin and cos :have We11

ayaxn

iii

n

iii

◊ Because (1) n is usually very large, (2) the individual

sin cos )( and 222 ryrAxyAxr

◊ Because (1) n is usually very large, (2) the individual amplitudes ai are random, and (3) the phases θi have a uniform distribution, it can be assumed that (from the central limit theorem) x and y are both Gaussian variables with means equal to zero and variance:

67

222 yx


◊ Rice distribution◊ Because x and y are independent random variables, the joint

distribution p(x,y) is

2

22

2 2exp

21)()(),(

yxypxpyxp

◊ The distribution p(r,θ) can be written as a function of p(x,y) :

yxpJrp ),(),(

rrr

yryxrx

J

cossinsincos

////

68

ryry cossin//



2

22

2exp

2),(

yxrrp

2

22

2)sin()cos(exp

2

22

rArr

2

22 cos2exp

22

ArArr

22

2

cose p1e p

2exp

2

ArArr

22 exp

22exp

69



(pdf)function density y probabilitahason distributi RiceThe

)()(

:bygiven 2

drprp

cos1

),()(

222

0

dArArr

drprp

otherwise 0

0 cosexp21

2exp

0222 rdArArr

70


◊ Nakagami m-distributiong◊ Frequently used to characterize the statistics of signals

transmitted through multi-path fading channels.◊ PDF is given by Nakagami (1960) as:

2 /12 2

m mrm

m

2

/12

RE

erm

rp mrmR

21 ,

][

22

2

mRE

m

RE

figurefadingthecalled moments, of ratio theas defined is mparameter The

2][ RE

71

figure.fading the


◊ Nakagami m-distributiong◊ The n-th moment of R is:

2/2/ n

2/n

mmnmRE

◊ By setting m=1, the PDF reduces to a Rayleigh PDF.y g

72


◊ Lognormal distribution:◊

.varianceandmean with ddistributenormally is where,lnLet

2

mXRX

◊ The PDF of R is given by:

. varianceand

2 2ln / 21 02

r me rp r r

◊ The lognormal distribution is suitable for modeling the effect

0 0r

g gof shadowing of the signal due to large obstructions, such as tall buildings, in mobile radio communications.

73


◊ Multivariate Gaussian distribution◊ Assume that Xi, i=1,2,…,n, are Gaussian random variables

with means mi, i=1,2,…,n; variances σi2, i=1,2,…,n; and

covariances μij, i,j=1,2,…,n. The joint PDF of the Gaussian random variables Xi, i=1,2,…,n, is defined as

xxnnxxxp mxMmxM

1

21exp

det21,...,, 2/12/21

◊ M denotes the n × n covariance matrix with elements {μij};◊ x denotes the n × 1 column vector of the random variables;◊ m denote the n × 1 column vector of mean values m i=1 2 n◊ mx denote the n × 1 column vector of mean values mi, i=1,2,…,n.◊ M-1 denotes the inverse of M.◊ x’ denotes the transpose of x.

74


◊ Multivariate Gaussian distribution (cont.)( )◊ Given v the n-dimensional vector with elements υi,

i=1,2,…,n, the characteristic function corresponding to the n-dimentional joint PDF is:

1j

Mvvvmv xv

21exp X

j jeEj

75


◊ Bi-variate or two-dimensional Gaussian◊ The bivariate Gaussian PDF is given by:

1 21,p x x

1 2 21 2

2 22 22 1 1 1 2 1 1 2 2 1 2 2

,2 1

2exp

p

x m x m x m x m

2 2 21 2

21 1 12

exp2 1

mE x m x m

m M 12 1 1 2 22

2 12 2

, ,

, , 0 1

X

ijij ij

E x m x mm

i j

m M

21 1 2

2

, ,

ij iji j

j

M

21 2 1 2

22 2 2

1, 1

M

76

21 2 2 22 2 2

1 2 11 2 1


◊ Bi-variate or two-dimensional Gaussian◊ ρ is a measure of the correlation between X1 and X2.◊ When ρ=0, the joint PDF p(x1,x2) factors into the product ρ , j p( 1, 2) p

p(x1)p(x2), where p(xi), i=1,2, are the marginal PDFs.◊ When the Gaussian random variables X1 and X2 are 1 2

uncorrelated, they are also statistically independent. This property does not hold in general for other distributions.

◊ This property can be extended to n-dimensional Gaussian random variables: if ρij=0 for i≠j, then the random variables Xi, i=1 2 n are uncorrelated and hence statisticallyi=1,2,…,n, are uncorrelated and, hence, statistically independent.

77

Upper Bounds on the Tail ProbabilityUpper Bounds on the Tail Probability

◊ Chebyshev inequalityy q y◊ Suppose X is an arbitrary random variable with finite mean mx

and finite variance σx2. For any positive number δ:

2

2

x

xmXP

◊ Proof:

2 2 2( ) ( ) ( ) ( )d d

2 2 2

| |

2 2

( ) ( ) ( ) ( )

( ) (| | ) x

x x xx m

x

x m p x dx x m p x dx

p x dx P X m

| |

( ) (| | )x

xx mp

78


◊ Chebyshev inequalityy q y◊ Another way to view the Chebyshev bound is working with

the zero mean random variable Y=X-mx.◊ Define a function g(Y) as:

E g Y g y p y dy 1 Y

g g y p y dy

p y dy p y dy P Y

2Y

1 Y

0 Yg Y

◊ Upper-bound g(Y) by the quadratic (Y/δ)2, i.e.

YYg

◊ The tail probability 2

2

2

2

2

2

2

2

xyYEYEYgE

79


◊ Chebychev inequalityy q y◊ A quadratic upper bound on g(Y) used in obtaining the tail

probability (Chebyshev bound)

◊ For many practical applications, the Chebyshev bound is

80

y p pp , yextremely loose.


◊ Chernoff bound◊ The Chebyshev bound given above involves the area under

the two tails of the PDF. In some applications we are interested only in the area under one tail, either in the interval (δ, ∞) or in the interval (-∞, δ).

◊ In such a case, we can obtain an extremely tight upper boundby over-bounding the function g(Y) by an exponential having a parameter that can be optimized to yield as tight an uppera parameter that can be optimized to yield as tight an upper bound as possible.

◊ Consider the tail probability in the interval (δ ∞)◊ Consider the tail probability in the interval (δ, ∞).

1 and is defined as

0 v Y Y δ

g Y e g Y g YY δ

81

where 0 is the parameter to be optimized.v

Upper Bounds on the Tail ProbabilityUpper Bounds on the Tail Probability◊ Chernoff bound

◊

◊ The expected value of g(Y) is

YveEYPygE

82

◊ This bound is valid for any υ 0.


◊ Chernoff bound◊ The tightest upper bound is obtained by selecting the value

that minimizes E(eυ(Y-δ)).◊ A necessary condition for a minimum is:

0YveEd 0eEdv

( ) ( )( ) ( )v Y v Yd dE E ( ) ( )

( )

( ) ( )

[( ) ]

v Y v Y

v Y

E e E edv dv

E Y e

[ ( ) ( )] 0v vY vYe E Ye E e

0 vYvY eEYeE Find

83

0 eEYeE Find


◊ Chernoff bound◊ v

:isyprobabilit tailsided-one on the boundupper thesolution, thebe ˆLet

Yvv eEeYP ˆˆ :isy probabilit

◊ An upper bound on the lower tail probability can be obtained in a similar manner, with the result that,

0 ˆˆ Yvv eEeYP

84


◊ Chernoff bound◊ Example: Consider the (Laplace) PDF p(y)=e-|y|/2.

◊ The true tail probability is:◊ The true tail probability is:

edyeYP y

21

21

85

22


◊ Chernoff bound◊ Example (cont.)

12 eEvYeE vYvY

02obtain we,0 Since

11

112

22

vveEYeE

vveE

vvYeE

vYvY

positive) bemust ˆ( 11ˆ

,2

vv

112 11

2

22

eYP

boundChebyshevfor 1 :1for

112

2

2

eYP

86

y2 2

Sums of Random Variables and the Central Limit Sums of Random Variables and the Central Limit TheoremTheorem

◊ Sum of random variables◊ Suppose that Xi, i=1,2,…,n, are statistically independent and

identically distributed (iid) random variables, each having a finite mean mx and a finite variance σx

2. Let Y be defined as the normalized sum, called the sample mean:

n

iiX

nY

1

1

◊ The mean of Y is

n

iXEmYE 1

x

iiy

m

XEn

mYE

1

87

Sums of Random Variables and the Central Limit Sums of Random Variables and the Central Limit TheoremTheorem

◊ Sum of random variables◊ The variance of Y is:◊ The variance of Y is:

mXXEmYEmYEσn n

ji222222 1

mXEXEXE

mXXEn

mYEmYEσ

n nn

i jxjixyy

22

1 12

11

mXEXEn

XEn

xi

jij

jii

i

2

1 12

12

11

nσmmnn

nmσn

n x

xxxx

222

222

2 111

◊ An estimate of a parameter (in this case the mean mx) that satisfies the conditions that its expected value converges to the true value of the parameter and the variance converges to

88

the true value of the parameter and the variance converges to zero as n→∞ is said to be a consistent estimate.

Stochastic ProcessesStochastic Processes

◊ Many of random phenomena that occur in nature are◊ Many of random phenomena that occur in nature are functions of time.

◊ In digital communications we encounter stochastic◊ In digital communications, we encounter stochastic processes in:

The characterization and modeling of signals generated by◊ The characterization and modeling of signals generated by information sources;

◊ The characterization of communication channels used to◊ The characterization of communication channels used to transmit the information;

◊ The characterization of noise generated in a receiver;◊ The characterization of noise generated in a receiver;◊ The design of the optimum receiver for processing the

received random signal.

89

g


◊ Introduction◊ Introduction◊ At any given time instant, the value of a stochastic process is

a random variable indexed by the parameter t. We denote y psuch a process by X(t).

◊ In general, the parameter t is continuous, whereas X may be either continuous or discrete, depending on the characteristics of the source that generates the stochastic process.

◊ The noise voltage generated by a single resistor or a single information source represents a single realization of theinformation source represents a single realization of the stochastic process. It is called a sample function.

90


◊ Introduction (cont.)( )◊ The set of all possible sample functions constitutes an

ensemble of sample functions or, equivalently, the stochastic process X(t).

◊ In general, the number of sample functions in the ensemble is assumed to be extremely large; often it is infinite.

◊ Having defined a stochastic process X(t) as an ensemble of l f ti id th l f thsample functions, we may consider the values of the process

at any set of time instants t1>t2>t3>…>tn, where n is any positive integerpositive integer.

◊ j ib h illi idh i

are ,,...,2,1, variablesrandom thegeneral,In i it nitXX

91

.,...,,PDFjoint by their lly statisticazedcharacteri21 nttt xxxp


◊ Stationary stochastic processes◊ Consider another set of random variables ,

1 2 where is an arbitrary time shift These randomit t in X X t t

i n t

1 2

1, 2,..., , where is an arbitrary time shift. These random

variables are characterized by the joint PDF , ,..., .nt t t t t t

i n t

p x x x

◊The jont PDFs of the random variables and 1 2 ,

may or may not be identical. When they are identical, i.e., wheni it t tX X ,i , ,...,n

1 2 1 2 , ,..., , ,...,

for all andn nt t t t t t t t tp x x x p x x x

t

all it is said to be stationary in the strict sensen◊ When the joint PDFs are different, the stochastic process is

non-stationary.

for all and t all , it is said to be stationary in the strict sense.n

92

y


◊ Averages for a stochastic process are called ensemble averages.◊

i

nn

t

dxxpxXE

Xn :as defined is variablerandom theof momentth The

◊

iiii tttt dxxpxXE

on thedependwillmomentth theofvalue thegeneral,In n

◊

.on depends of PDF theif instant timepg ,

iti tXti

allforstationaryisprocessWhen the txpxp

timeoftindependenismomentththeeconseq enca as and, time,oft independen is PDF theTherefore,

.allfor ,stationaryisprocess When the txpxpii ttt

time.oft independenismomentth thee,consequenc n

93


◊ Two random variables: , 1, 2.it iX X t i

◊ The correlation is measured by the joint moment:i

,t t t t t t t tE X X x x p x x dx dx

◊ Since this joint moment depends on the time instants t1 and

t2, it is denoted by φ(t1 ,t2).

1 2 1 2 1 2 1 2,t t t t t t t tp

,

◊ φ(t1 ,t2) is called the autocorrelation function of the stochastic process.

◊ For a stationary stochastic process, the joint moment is:

1 2 1 2 1 2( ) ( , ) ( ) ( )t tE X X t t t t

◊

2

' '1 1 1 1 1 1

( ) ( ) ( ) ( ) ( )t t t t t tE X X E X X E X X

(Even Function)

94

◊ Average power in the process X(t): φ(0)=E(Xt2).


◊ Wide-sense stationary (WSS)◊ Wide sense stationary (WSS)◊ A wide-sense stationary process has the property that the

mean value of the process is independent of time (a p p (constant) and where the autocorrelation function satisfies the condition that φ(t1,t2)=φ(t1-t2).

◊ Wide-sense stationarity is a less stringent condition than strict-sense stationarity.

◊ If not otherwise specified, any subsequent discussion in which correlation functions are involved, the less stringent condition (wide sense stationarity) is impliedcondition (wide-sense stationarity) is implied.

95


◊ Auto-covariance function◊ Auto covariance function◊ The auto-covariance function of a stochastic process is

defined as:

1 21 2 1 2

1 2 1 2

,

,

t tt t E X m t X m t

t t m t m t

◊ When the process is stationary, the auto-covariance function simplifies to:

1 2 1 2,t t m t m t

p

◊ For a Gaussian random process, higher-order moments can 2

1 2 1 2,t t t t m (function of time difference)

p , gbe expressed in terms of first and second moments. Consequently, a Gaussian random process is completely h i d b i fi

96

characterized by its first two moments.


◊ Averages for a Gaussian process◊ Suppose that X(t) is a Gaussian random process. At time

instants t=ti, i=1,2,…,n, the random variables Xti, i=1,2,…,n, are jointly Gaussian with mean values m(t ) i=1 2 n andare jointly Gaussian with mean values m(ti), i=1,2,…,n, and auto-covariances:

, , , 1, 2,..., .i j i jt t E X m t X m t i j n ◊ If we denote the n × n covariance matrix with elements μ(ti,tj)

by M and the vector of mean values by mx, the joint PDF of

, , , 1, 2,..., .i ji j t i t jt t E X m t X m t i j n

y y x, jthe random variables Xti, i=1,2,…,n, is given by:

xxxp mxMmx 11exp1

◊ If the Gaussian process is wide-sense stationary, it is also

xxnnxxxp mxMmx

M 2exp

det2,...,, 2/12/21

97

strict-sense stationary.


◊ Averages for joint stochastic processesg j p◊ Let X(t) and Y(t) denote two stochastic processes and let

Xti≡X(ti), i=1,2,…,n, Yt’j≡Y(t’j), j=1,2,…,m, represent the

i j jrandom variables at times t1>t2>t3>…>tn, and t’1>t’2>t’3>…>t’m , respectively. The two processes are h t i d t ti ti ll b th i j i t PDFcharacterized statistically by their joint PDF:

Th l i f i f X( ) d Y( ) d t d b ' ' '1 2 1 2

, ,..., , , ,...,n m

t t t t t tp x x x y y y

◊ The cross-correlation function of X(t) and Y(t), denoted by φxy(t1,t2), is defined as the joint moment:

)()()( ddYXEtt◊ The cross-covariance is:

21212121),()(),( 21 ttttttttxy dydxyxpyxYXEtt

)()()()( tmtmtttt

98

)()(),(),( 212121 tmtmtttt yxxyxy


◊ Averages for joint stochastic processesg j p◊ When the process are jointly and individually stationary, we

have φxy(t1,t2)=φxy(t1-t2), and μxy(t1,t2)= μxy(t1-t2):y y y y

◊ The stochastic processes X(t) and Y(t) are said to be

' ' ' '1 1 1 1 1 1( ) ( ) ( ) ( ) ( )xy t t yxt t t t

E X Y E X Y E Y X

◊ The stochastic processes X(t) and Y(t) are said to be statistically independent if and only if :

)()()( '''''' yyypxxxpyyyxxxp

for all choices of ti and t’i and for all positive integers n and m.Th id b l d if

),...,,(),...,,(),...,,,,...,,( ''2

'121''

2'121 mnmn tttttttttttt yyypxxxpyyyxxxp

◊ The processes are said to be uncorrelated if

)()()( YEXEtt 0)( 21 tt

99

)()(),(2121 ttxy YEXEtt 0),( 21 ttxy

Stochastic ProcessesStochastic Processes◊ Complex-valued stochastic process

◊ A complex valued stochastic process Z(t) is defined as:◊ A complex-valued stochastic process Z(t) is defined as:

where X(t) and Y(t) are stochastic processes)()()( tjYtXtZ

where X(t) and Y(t) are stochastic processes.◊ The joint PDF of the random variables Zti≡Z(ti), i=1,2,…,n, is

given by the joint PDF of the components (X Y ) i=1 2 ngiven by the joint PDF of the components (Xti, Yti), i=1,2,…,n. Thus, the PDF that characterizes Zti, i=1,2,…,n, is:

),...,,,,...,,( tttttt yyyxxxp◊ The autocorrelation function is defined as:

), ,,,, ,,(2121 nn tttttt yyyp

1 1

1 2 1 1 2 21 21 1( , ) ( )2 21 ( ) ( ) ( ) ( )

zz t t t t t tt t E Z Z E X jY X jY

t t t t j t t t t

(**)

100

1 2 1 2 1 2 1 2 ( , ) ( , ) ( , ) ( , ) 2 xx yy yx xyt t t t j t t t t


◊ Averages for joint stochastic processes:g j p◊ When the processes X(t) and Y(t) are jointly and individually

stationary, the autocorrelation function of Z(t) becomes:

)()(),( 2121 zzzzzz tttt

1◊ φZZ(τ)= φ*

ZZ(-τ) because from (**):1 21 2

1( , ) ( )2zz t tt t E Z Z

' ' ' '1 1 1 1 1 1

1 1 1( ) ( ) ( ) ( ) ( )2 2 2zz t t zzt t t t

E Z Z E Z Z E Z Z

101


◊ Averages for joint stochastic processes:◊ Suppose that Z(t)=X(t)+jY(t) and W(t)=U(t)+jV(t) are two

complex-valued stochastic processes. The cross-correlation functions of Z(t) and W(t) is defined as:functions of Z(t) and W(t) is defined as:

1 21 21( , ) ( )2zw t tt t E Z W

1 1 2 2

1 21

t t t tE X jY U jV

◊ When X(t), Y(t),U(t) and V(t) are pairwise-stationary, the 1 2 1 2 1 2 1 2

1 ( , ) ( , ) ( , ) ( , ) 2 xu yv yu xvt t t t j t t t t

cross-correlation function become functions of the time difference.

◊1 1 1( ) ( ) ( ) ( ) ( )E Z W E Z W E W Z

102

◊ ' ' ' '1 1 1 1( ) ( ) ( ) ( ) ( )

2 2 2i izw t t wzt t t t

E Z W E Z W E W Z

Power Density SpectrumPower Density Spectrumy py p

◊ A signal can be classified as having either a finite (nonzero) average power (infinite energy) or finite energy.

◊ The frequency content of a finite energy signal is obtained q y gy gas the Fourier transform of the corresponding time function.

◊ If the signal is periodic, its energy is infinite and, consequently its Fourier transform does not exist Theconsequently, its Fourier transform does not exist. The mechanism for dealing with periodic signals is to represent them in a Fourier seriesthem in a Fourier series.

103

Power Density SpectrumPower Density Spectrum

◊ A stationary stochastic process is an infinite energy y p gysignal, and, hence, its Fourier transform does not exist.

◊ The spectral characteristic of a stochastic signal is◊ The spectral characteristic of a stochastic signal is obtained by computing the Fourier transform of the autocorrelation function.autocorrelation function.

◊ The distribution of power with frequency is given by the function:the function:

def fj 2

◊ The inverse Fourier transform relationship is:

dff fj 2

104

dfef fj 2


◊ 00 2

tXEdff◊ Since φ(0) represents the average power of the stochastic signal,

which is the area under Φ(f ), Φ(f ) is the distribution of power as

a function of frequency.◊ Φ( f ) is called the power density spectrum of the stochastic

process.◊ If the stochastic process is real, φ(τ) is real and even, and, hence Φ( f ) is real and e en

(from definition)

(P.94)( t f d fi iti )Φ( f ) is real and even.

◊ If the stochastic process is complex, φ(τ)=φ*(-τ) and Φ( f ) is real because:

( )

(P.101)

(easy to prove from definition)

real because:

* * 2 * 2 '

2

' 'j f j f

j f

f e d e d

d f

( )

105

2 j fe d f


◊ Cross-power density spectrump y p◊ For two jointly stationary stochastic processes X(t) and Y(t),

which have a cross-correlation function φxy(τ), the Fourier ytransform is:

def fjxyxy

2

◊ Φxy( f ) is called the cross-power density spectrum.◊ dedef fj

xyfj

xyxy

2*2**

fde yxfj

yx

yyy

2

◊ If X(t) and Y(t) are real stochastic processes

* 2j ff e d f

f f →

106

xy xy xyf e d f

yx xyf f →

Response of a Linear TimeResponse of a Linear Time--Invariant System to a Invariant System to a Random Input SignalRandom Input Signal

◊ Consider a linear time-invariant system (filter) that is characterized by its impulse response h(t) or equivalently, by its

p gp g

frequency response H( f ), where h(t) and H( f ) are a Fourier transform pair. Let x(t) be the input signal to the system and let y(t) denote the output signaly(t) denote the output signal.

y t h x t d

◊ Suppose that x(t) is a sample function of a stationary stochastic process X(t). Since convolution is a linear operation performed on the input signal x(t) the expected value of the integral ison the input signal x(t), the expected value of the integral is equal to the integral of the expected value.

ym E Y t h E X t d

0

y

x xm h d m H

stationary

107

◊ The mean value of the output process is a constant.


◊ The autocorrelation function of the output is:

p gp g

YYEtt ttyy*

21 21,

21

ddtXtXEhh 2

*1

*

21

ddtthh xx 21

*

◊ If the input process is stationary, the output is also stationary:

*yy xxh h d d

108


◊ The power density spectrum of the output process is:

p gp g

2j fyy yyf e d

* 2j fxxh h e d d d

(by making τ0=τ+α-β) 2

xx f H f

( y g 0 β)◊ The power density spectrum of the output signal is the product of

the power density spectrum of the input multiplied by the magnitude squared of the frequency response of the system.

109


◊ When the autocorrelation function φyy(τ) is desired, it is usually

p gp g

yyeasier to determine the power density spectrum Φyy(f ) and then to compute the inverse transform.

2yy

j fyy f e df

2 2j ff H f e df

◊ The average power in the output signal is:

xxj ff H f e df

2xx0yy f H f df

◊ Since φyy(0)=E(|Yt|2) , we have:

20f H f df

valid for any H( f )

110

xx 0f H f df valid for any H( f ).


◊ Suppose we let |H( f )|2=1 for any arbitrarily small interval f1 ≤ f ≤

p gp g

f2 , and H( f )=0 outside this interval. Then, we have:

2 0f

xx dff

This is possible if an only if Φxx( f )≥0 for all f.

1f

xx ff

◊ Conclusion: Φxx( f )≥0 for all f.

111


◊ Cross-correlation function between y(t) and x(t)

p gp g

◊ 21

21, 2

*1

*21 21

dtXtXEhXYEtt ttyx

i f

t tij i tldt h tiTh

2121

tYtX

ttdtth yxxx

Function of t1-t2

◊ With t1-t2=τ, we have:

.stationaryjointly are andprocessesstochastic The tYtX

1 2 ,

dh xxyx

◊ In the frequency domain, we have:

fHff xxyx

112

fff xxyx

DiscreteDiscrete--Time Stochastic Signals and SystemsTime Stochastic Signals and Systems

◊ Discrete-time stochastic process X(n) consisting of an ensemble

g yg y

of sample sequences {x(n)} are usually obtained by uniformly sampling a continuous-time stochastic process.

◊ The mth moment of X(n) is defined as:

nnmn

mn dXXpXXE

◊ The autocorrelation sequence is:

nnnn p

knknknkn dXdXXXpXXXXEkn ,

21

21, **

◊ The auto-covariance sequences is:

*1,, kn XEXEknkn

113

2

,, knknkn


◊ For a stationary process, we have φ(n,k)≡φ(n-k), μ(n,k)≡μ(n-k), d

g yg y

and 2

21

xmknkn

where mx=E(Xn) is the mean value.◊ A discrete-time stationary process has infinite energy but a◊ A discrete time stationary process has infinite energy but a

finite average power, which is given as:

2 0E X

◊ The power density spectrum for the discrete-time process is obtained by computing the Fourier transform of φ(n)

0nE X

obtained by computing the Fourier transform of φ(n).

fnjenf 2

114

n

f


◊ The inverse transform relationship is:

g yg y

◊ The inverse transform relationship is:

21 2 dfefn fnj

◊ The power density spectrum Φ( f ) is periodic with a

21ff

period fp=1. In other words, Φ( f+k)=Φ( f ) for k=0,±1,±2,….

◊ The periodic property is a characteristic of the Fourier transform of any discrete-time sequence.

115


◊ Response of a discrete-time, linear time-invariant

g yg y

p ,system to a stationary stochastic input signal.◊ The system is characterized in the time domain by its unit y y

sample response h(n) and in the frequency domain by the frequency response H( f ).

n

fnjenhfH 2

◊ The response of the system to the stationary stochastic input signal X(n) is given by the convolution sum:

knxkhny

116

k


◊ Response of a discrete-time, linear time-invariant

g yg y

p ,system to a stationary stochastic input signal.◊ The mean value of the output of the system is:p y

yk

m E y n h k E x n k

0

k

x xk

m h k m H

(P. 107)

where H(0) is the zero frequency [direct current (DC)] gain of the system.

k

y

117

DiscreteDiscrete--Time Stochastic Signals and SystemsTime Stochastic Signals and Systems◊ The autocorrelation sequence for the output process is:

1k E k

g yg y

12

1

yy k E y n y n k

h i h j E x n i x n k j

2 i j

h i h j E x n i x n k j

h i h j k j i

◊ By taking the Fourier transform of φ (k) we obtain the

xxi j

h i h j k j i

◊ By taking the Fourier transform of φyy(k), we obtain the corresponding frequency domain relationship:

2f f H f

◊ Φyy( f ), Φxx( f ), and H( f ) are periodic functions of

yy xxf f H f (P. 109)

118

◊ Φyy( f ), Φxx( f ), and H( f ) are periodic functions of frequency with period fp=1.

CCyclostationaryyclostationary ProcessesProcesses

◊ For signals that carry digital information, we encounter h i i h i i l h i distochastic processes with statistical averages that are periodic.

◊ Consider a stochastic process of the form:

nn

X t a g t nT

where {an} is a discrete-time sequence of random variables with mean ma=E(an) for all n and autocorrelation sequence φ (k)=E(a* a )/2φaa(k)=E(a*nan+k)/2.

◊ The signal g(t) is deterministic.◊ The sequence {a } represents the digital information sequence◊ The sequence {an} represents the digital information sequence

that is transmitted over the communication channel and 1/Trepresents the rate of transmission of the information symbols.

119

ep ese ts t e ate o t a s ss o o t e o at o sy bo s.

CyclostationaryCyclostationary ProcessesProcesses

◊ The mean value is:

nn

E X t E a g t nT

The mean is time varying and it is periodic with period T

an

m g t nT

The mean is time-varying and it is periodic with period T.

◊ The autocorrelation function of X(t) is:

12

1

,xx t t E X t X t

E T T

12 n m

n m

E a a g t nT g t mT

T T

120

aan m

m n g t nT g t mT


◊ We observe that

for k=±1,±2,…. Hence, the autocorrelation function of X(t) is also , ,xx xxt kT t kT t t

periodic with period T.◊ Such a stochastic process is called cyclostationary or periodically

stationary.◊ Since the autocorrelation function depends on both the variables t

d i f d i i i h fand τ, its frequency domain representation requires the use of a two-dimensional Fourier transform.The ti t l ti f ti over a single period is◊ The time-average autocorrelation function over a single period is defined as:

2

2

1 ,T

xx xxTt t dt

T

121

2xx xxTT


◊ Thus, we eliminate the tie dependence by dealing with the average autocorrelation function.

◊ The Fourier transform of φxx(τ) yields the average power density spectrum of the cyclostationary stochastic process.

◊ This approach allows us to simply characterize cyclostationaryi th f d i i t f th tprocess in the frequency domain in terms of the power spectrum.

◊ The power density spectrum is:

2j fxx xxf e d

122

DC-02-Probability and Stochastic Processes

Documents

Transcript of DC-02-Probability and Stochastic Processes