Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 1 of 54 ECE 3800
Henry Stark and John W. Woods, Probability, Statistics, and Random Variables for Engineers, 4th ed.,
Pearson Education Inc., 2012. ISBN: 978-0-13-231123-6
Chapter 8 Random Sequences
Sections 8.1 Basic Concepts 442 Infinite-length Bernoulli Trials 447 Continuity of Probability Measure 452 Statistical Specification of a Random Sequence 454 8.2 Basic Principles of Discrete-Time Linear Systems 471 8.3 Random Sequences and Linear Systems 477 8.4 WSS Random Sequences 486 Power Spectral Density 489 Interpretation of the psd 490 Synthesis of Random Sequences and Discrete-Time Simulation 493 Decimation 496 Interpolation 497 8.5 Markov Random Sequences 500 ARMA Models 503 Markov Chains 504 8.6 Vector Random Sequences and State Equations 511 8.7 Convergence of Random Sequences 513 8.8 Laws of Large Numbers 521 Summary 526 Problems 526 References 541
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 2 of 54 ECE 3800
8.1 Basic Concepts
A random process is a collection of time functions and an associated probability description.
When a continuous or discrete or mixed process in time/space can be describe mathematically as a function containing one or more random variables.
A sinusoidal waveform with a random amplitude. A sinusoidal waveform with a random phase. A sequence of digital symbols, each taking on a random value for a defined time period
(e.g. amplitude, phase, frequency). A random walk (2-D or 3-D movement of a particle)
The entire collection of possible time functions is an ensemble, designated as tx , where one
particular member of the ensemble, designated as tx , is a sample function of the ensemble. In general only one sample function of a random process can be observed!
Think of: 20,sin twAtX
where A and w are known constants.
Note that once a sample has been observed … 111 sin twAtx
the function is known for all time, t.
Note that, 2tx is a second time sample of the same random process and does not provide any “new information” about the value of the random variable.
221 sin twAtx
There are many similar ensembles in engineering, where the sample function, once known, provides a continuing solution. In many cases, an entire system design approach is based on either assuming that randomness remains or is removed once actual measurements are taken!
For example, in communications there is a significant difference between coherent (phase and frequency) demodulation and non-coherent (i.e. unknown starting phase) demodulation.
On the other hand, another measurement in a different environment might measure 21212 sin twAtx
In this “space” the random variables could take on other values within the defined ranges. Thus an entire “ensemble” of possibilities may exist based on the random variables defined in the random process.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 3 of 54 ECE 3800
For example, assume that there is a known AM signal transmitted:
twtAbts sin1
at an undetermined distance the signal is received as
20,sin1 twtAbty
The received signal is mixed and low pass filtered …
20,cossin1cos twtwtAbthtwtythtx
20,sin2sin5.01cos twtAbthtwtythtx
If the filter removes the 2wt term, we have
20,sin2
1cos
tAbtwtythtx
Notice that based on the value of the random variable, the output can change significantly! From producing no output signal, ( ,0 ), to having the output be positive or negative ( 20 toorto ). P.S. This is not how you perform non-coherent AM demodulation.
To perform coherent AM demodulation, all I need to do is measured the value of the random variable and use it to insure that the output is a maximum (i.e. mix with mtw cos , where.
1tm
Note: the phase is a function of frequency, time, and distance from the transmitter.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 4 of 54 ECE 3800
From our textbook
Random Stochastic Sequence
Definition 8.1-1. Let P,, be a probability space. Let . Let ,nX be a mapping of the sample space into a space of complex-valued sequences on some index set Z. If, for each fixed integer Zn , ,nX is a random variable, then ,nX is a ransom (stochastic) sequence. The index set Z is all integers, n , padded with zeros if necessary,
Example sets of random sequences.
Figure 8.1-1 Illustration of the concept of random sequence X(n,ζ), where the ζ domain (i.e., the sample space Ω) consists of just ten values. (Samples connected only for plot.)
The sequences can be thought of as “realizations” of the random sequence or sample sequences.
The absolute sequence is the realization of individual random variables in time.
One the realization exists; it becomes statistical data related to one instantiation of the Random Sequence.
Prior to collecting a realization, the Random Sequence can be defined probabilistically.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 5 of 54 ECE 3800
Example 8.1-1&2
Separable random sequences may be constructed by combining a deterministic sequence with one or more random variables.
The classic example already shown is a sinusoid with random amplitude and phase:
nAnX
20
12sin,
Where the amplitude and phase are R.V. defined based on the probability space selected.
Example8.1‐3
A random sequence with finite support is limited in time or samples.
else
NnXnX n
,0
1,,
Definition8.1‐2IndependentRandomSequence
An independent random sequence is one whose random variables at any time are jointly independent for all positive integers.
jifor
jiforXXE ji
,
,2
22
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 6 of 54 ECE 3800
Figure 8.1-3 Example of a sample sequence of a random sequence. (Samples connected only for plot.)
Figure 8.1-4 Close-up view of portion of sample sequence.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 7 of 54 ECE 3800
Infinite-length Bernoulli Trials p. 447-448.
Defining an infinite space consisting of all possible sequential outcomes of the R.V over a defined interval. It also defines the probability of the infinite space as the product of the probability that generated one such instantiation.
nnX 1
the infinite cross product that the points in the infinite space consist of all the infinite-length sequences of events.
Example8.1‐5:Buildingcorrelatednoise
For a Bernoilli random sequence W(m) just described, We form the “filtered” output
n
m
mn nformWanX1
1,
For the Bernoulli random sequence, we know that pmWP 1 and pmWP 10 .
We can define the mean of the output as
n
m
mnn
m
mn mWEamWaEnXE11
1
0
1
11
n
m
mnn
m
mnn
m
mn aapaapapanXE
a
ap
a
ap
a
aapanXE
nnnn
1
1
1
1
1
11
1
The new sequence created involved the sum of random variables and as such is a random sequence; however, the individual elements are no longer IID.
We compute the cross-product of the first two terms:
12112 WWWaEXXE
12112 2 WWEWEaXXE
pppaXXE 12
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 8 of 54 ECE 3800
Note that
pa
apXE
1
11
1
and apa
apXE
11
12
2
Therefore,
21112 2 XEXEappppaXXE
The outputs are correlated (as you might expect).
The variance of the new sequence can be computed
n
m
mnn
m
mn mWVaramWaVarnXVar1
2
1
ppa
aqp
a
anXVar
nn
11
1
1
12
2
2
2
The random outputs can be generated recursively as
n
m
mn nformWanX1
1,
1
1
10 1,n
m
mn nformWaanWanX
1,1 nfornXanWnX
Then
Figure 8.1-5 A feedback filter that generates correlated noise X[n] from an uncorrelated sequence W[n].
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 9 of 54 ECE 3800
Matlab simulation of the autoregressive sequence
Data Generation numsamples = 400; p = 0.50; % X(0) prob (1-p) x(1) prob p q = 1-p; x = zeros(numsamples,1); u = rand(numsamples,1); w = double(u>=q);
As a Bernoulli R.V. p and q are defined as well as defining P[0]=q and P[1]=p.
The Auto-Regressive (AR) Sequence is generated as % The autoregressive function alpha = 0.95; x(1) = w(1); for ii=2:numsamples x(ii) = alpha*x(ii-1)+w(ii); end % The IIR filter implementation b = 1.0; a = [1.0 -alpha]; xx = filter(b,a,w);
It can be done literally based on the regressive equation or as an IIR filter …
w(n
)
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 10 of 54 ECE 3800
Knowing something about an IIR filter, the maximum output for a unity gain input is expected to be ….
2095.01
1
1
11
1
aanX
n
m
mn
With mean value of the input of 0.5, I would expect results to be less than 20 and possible randomly moving about 10 …
Note that the R.V. statistics are generated from the data for comparison purposes …
pnWE
ppnWVar 1
a
apnXE
n
1
1
ppa
anXVar
n
11
12
2
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 11 of 54 ECE 3800
Continuity of Probability Measure 452
Relating to Countable Additively …
N
nn
N
nn APAP
11
for a mutually exclusive infinite collection of events.
For an increasing sequence of events where each Bn includes all previous Bj for j<n. or
11 BBB nn
We can define
N
nnBB
1
and define proper disjoint sets such that
11 BA and Cnnn BBA 1
Then
N
nn
N
nn
N
nn APAPBPBP
111
This is a total probability statement and a statement of subdividing the inclusive area B into disjoint subsets.
Inverse Corollary
For decreasing sequence of events where each Bn is a subset of all previous Bj for j<n. or
11 BBB nn
N
nnBB
1
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 12 of 54 ECE 3800
Example 8.1-8 CDF continuity from the right at step discontinuities …
Using the decreasing sequence of events concept …
11 BBB nn
For a CDF xFn
xF XXn
1lim
But for every successive value of n, we can define
nxXBn
1:
With
N
nnBPBP
1
Leading to
xFBPBPn
xF Xnn
Xn
lim1
lim
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 13 of 54 ECE 3800
Statistical Specification of a Random Sequence
In general we are looking developing properties for developing random processes where:
(1) The statistical specification for the random sequence matches the probabilistic (or axiomatic) specification for the random variables used to generate the sequence.
(2) We will be interested in stationary sequences where the statistics do not change in time. We will be defining a wide-sense stationary random process definition where only the mean and the variance need to be constant in time.
A random sequence X[n] is said to be statistically specified by knowing its Nth-order CDFs for all integers N>=1, and for all times, n, n+1, …,n+N-1. That states that we know …
11
11
1,,1,
1,,1,;,,,
Nnnn
NnnnX
xNnXxnXxnXP
NnnnxxxF
If we specify all these infinite-order joint distributions at all finite times, using continuity of the probability measures, we can calculate the probabilities of events involving an infinite number of random variables via limiting operations involving the finite order CDFs.
Consistency can be guaranteed by construction … constructing models of stochastic sequences and processes.
Moments play an important role and, for Ergodic Sequences, they can be estimated from a single sample sequence of the infinite number that may be possible.
Therefore,
nnXn
XX
dxxfx
dxnxfxnXEn ;
and for a discrete valued random sequences
k
kkX xnXPxnXEn
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 14 of 54 ECE 3800
The Autocorrelation Function
The expected value of a random sequence evaluated at offset times can be determined.
lklkXlkKK dxdxxxfxxlXkXElkR ,, *
For sequences of finite average power … 2kXE , then the correlation function will exist.
We can also describe the “centered” autocorrelation sequence as the autocovariance.
*, llXkkXElkK XXKK
Note that
**
****
*,
lklXkXE
lklXklkXlXkXE
llXkkXElkK
XX
XXXX
XXKK
*,, lklkRlkK XXKKKK
Basicpropertiesofthefunction:
Hermitian symmetry *., klRlkR KKKK
Hermitian symmetry *., klKlkK KKKK
Deriving other functions
2, kXEkkRKK
2, XKK kkK
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 15 of 54 ECE 3800
Example 8.1-1 & 10 functions consisting of R.V and deterministic sequences
Let nfXnX ,
where X is a random variable and f is a deterministic function in sample time n.
Note then,
nfnfXEnXE X ,
The autocorrelation function becomes
dxxflfXkfXlXkXElkR XKK**,,,
dxxfXXlfkflkR XKK**,
**, XXElfkflkRKK
If X is a real R.V.
22*, XXKK lfkflkR
Similarly
**, XXXX XXElfkflkK
2*, XXX lfkflkK
Example8.1‐11Waitingtimesinaline
consider the random sequence of IID “exponential random variable” waiting times in a line. Assume that each of the waiting times per individual t(k) is based on the exponential.
0,exp
0,0;
tt
ttfntf
The waiting time is then described as
n
k
knT1
where 11 T , 212 T , … , nnT 21
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 16 of 54 ECE 3800
This calls for a summation of random variables, where the new pdf for each new sum is the convolution of the exponential pdf with the previous pdf or
tftftf 2;
tftftftftftf 2;3;
Exam #1 derived the first convolution
t
dttf0
expexp2;
ttdttft
exp1exp2; 2
0
2
Repeating
t
dttf0
2 expexp3;
tdttft
exp2
exp3;2
3
0
3
If you see the pattern …. we can jump to the nth summation where
tn
tn
ntfnn
n
exp!1
exp!1
;11
This is called the Erlang probability density function …
It is used to determine waiting times in lines and software queues … how long until your internet request can be processed!
The mean and variance of the Erlang pdf is
nnT
2
2
n
VarnT
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 17 of 54 ECE 3800
Gaussian Random Sequences
There are a few developments to consider: The sequence of Gaussian R.V. 0, WnW
The sum of two Gaussians R.V. 1 nWnWnX
ArandomsequencebasedontheGaussian.Assume iid Gaussian R.V. with zero mean and a variance
For 0 WnWE and 22WnWE
What about the autocorrelation
lk
lkkWElWkWElkR
W
WWWKK
0
,,
2
2222*
or it can be written as
lklkR WKK 2,
ArandomsequencebasedonthesumoftwoGaussians.
Assume iid Gaussian R.V. with zero mean and a variance
For 1 nWnWnX
Then, 021 WWWnWnWEnXE
and
2
222
22
22
22
2
2
12
112
1
W
WWW
WW nWEnWE
nWnWnWnWE
nWnWEnXE
also
*11, lWlWkWkWElkRKK
**** 1111, lWkWlWkWlWkWlWkWElkRKK But then
1111, 2222 lklklklklkR WWWWKK
112, 222 lklklklkR WWWKK
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 18 of 54 ECE 3800
Note that this is a preferred structure for an autocorrelation function …
112, 222 lklklklkRlkR WWWKKKK
which can be written as
112 222 nnnnRlkR WWWKKKK
The result shows that the time samples are not important individually, only the distance between the time samples!
Example8.1‐13RandomWalk
Continuing with infinite length Bernoulli trials, we will define taking steps either forward or backward and see if there is any progress in time.
Therefore let
Tailss
HeadsskW
,
,
n
k
kWnX1
The current position is based on the sum of time history of heads and tails. Let,
snksknsksrnX 2
Where there would be k heads and n-k tails. Based on the current position, we can “solve for k” in the Bernoulli equation as
2
nrk
But k is based on the Binomial R.V.
knkk pp
k
np
1 , for nk ,,2,1,0
else
nrrnpprnn
rnPsrnXP
rnrn
,0
withintegeran2,122
22
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 19 of 54 ECE 3800
We can find the mean and variance
spnsqspkWEnXEn
k
n
k
1211
Assuming equal probability
else
nrrnrn
nrn
PsrnXP
,0
withintegeran2,222
2
012 spnnXE
For W[n] IID
n
k
n
kjj
n
k
n
j
n
k
jWkWkWEjWkWEnXE1
11
2
11
2
n
k
n
kjj
n
k
jWEkWEkWEnXE1
11
22
22
1
222W
n
k
nnsqspnXE
222 snsqpnnXE
If we define a “normalized” random variable
nXn
nX 1
The normalized random variable based on the Central Limit Theorem would be equivalent to a Gaussian Normal with standard deviation s!
A probability bound on the range of X(n) could then be defined as …
s
a
s
bbnXaP
and
s
a
s
bbnnXanPbnXaP
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 20 of 54 ECE 3800
Unique properties of Erlang and Random Walk
The Erlang waiting time and Random Walk have a property called “independent-increments”, were the increments in value could be considered jointly independent. Unfortunately, the mean, the variance or both varied based on the time increments.
For many applications, we prefer that the probability is non-time or sample varying … that the moments are stationary.
Non-stationary means and variances are considerably harder to deal with in signal processing …
This is indicative of classes of sequences and processes … those that are stationary and those the vary as time or the number of samples increases.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 21 of 54 ECE 3800
Stationary vs. Nonstationary Random Processes
The probability density functions for random variables in time have been discussed, but what is the dependence of the density function on the value of time, t or n, when it is taken?
If all marginal and joint density functions of a process do not depend upon the choice of the time origin, the process is said to be stationary (that is it doesn’t change with time). All the mean values and moments are constants and not functions of time!
For nonstationary processes, the probability density functions change based on the time origin or in time. For these processes, the mean values and moments are functions of time.
In general, we always attempt to deal with stationary processes … or approximate stationary by assuming that the process probability distribution, means and moments do not change significantly during the period of interest.
Examples:
Resistor values (noise varies based on the local temperature)
Wind velocity (varies significantly from day to day)
Humidity (though it can change rapidly during showers)
The requirement that all marginal and joint density functions be independent of the choice of time origin is frequently more stringent (tighter) than is necessary for system analysis.
A more relaxed requirement is called stationary in the wide sense: where the mean value of any random variable is independent of the choice of time, t, and that the correlation of two random variables depends only upon the time difference between them. That is
XXtXE and
XXRXXttXXEtXtXE 00 1221 for 12 tt
You will typically deal with Wide-Sense Stationary Signals.
See Definition 8.1-5, p. 464, for the text definition of stationary. The definition of wide-sense stationary (WSS) random sequence also appears on p. 465, where the mean and the covariance of a random sequence are not based on time or sample number.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 22 of 54 ECE 3800
Stationary Systems Properties
MeanValue
00;; XXXX dxxfxdxnxfxnXEn
The mean value is not dependent upon the sample in time.
Autocorrelation
nlnkRnlXnkXElkR
dxdxnlnkxxfxxlkR
dxdxlkxxfxxlXkXElkR
KKKK
nlnknlnkXnlnkKK
lklkXlkKK
,.,
,;,,
,;,,
*
*
**
And in particular kRkRlkRlkRlkR KKKKKKKKKK 0,0,,
Autocovariance
nlnkKnlXnkXE
lXkXEllXkkXElkK
KKXX
XXXXKK
,.
,*
**
And in particular kKkKlkKlkKlkK KKKKKKKKKK 0,0,,
The autocorrelation and autocovariance are functions of the time difference and not the absolute time.
Of course as an example the text starts with something confusing and difficult …
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 23 of 54 ECE 3800
Example 8.1-15 A sequence with memory ….
A two random state function.
Figure 8.1-14 State-transition diagram of two-state (binary) random sequence with memory
Let akX and bkX ~
pqPwithnX
pPwithnXnX
1,1~
,1
Mean value
bqapXqXpEXE 0~01
aqbpXqXpEXE 00~1~
aqbqpapXqXpEXE 22 21~12
bqaqpbpXqXpEXE 22 211~2~
bqaqpbpqaqpbqpapXqXpEXE 322223 222~23
bpqqpqaqpqppXqXpEXE 223223 222~23
12~23 22 XEbqapbqpqaqppXqXpEXE
1~22~3~ XEaqbpXqXpEXE
The expected value “oscillates” from even to odd and back again …
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 24 of 54 ECE 3800
There is a binomial growth of probabilities based on the initial state chosen. When N is odd, all products of two transitions or an integer number time two transitions will be a. All others will be b.
As an interesting note … when p=1/2 and N is odd, 2
baNXE
abaqbpoddXEXE 2
1
2
11
Furthermore
baabaevenXEXE 2
1
2
1
4
1
2
1
4
12
The number of possible “even” transitions in state due to p and q are equal to the number of “odd” transitions in state due to p and q probabilities. But we know this from the Binomial pmf!
1,5,10,10,5,15
1,3,3,13
:
kkk
N
Example 8.1-16 A sequence with memory …. Autocorrelation
*, knXkXEknkRKK
knkabPabknkbaPba
knkaaPaaknkbbPbbknkRKK ,;,,;,
,;,,;,,
Note if we set a=0 this greatly simplifies ...
knkbbPbbknkRKK ,;,,
As a sequence of events, for the second event following the first, we can deal with conditional probability … given X[k]=b then
kbnkbPkbPbbknkbbPbbknkRKK ;|;;,;,,
or
bkXbnkXPbkXPbbknkRKK |,
Based on the mean value discussion 5.0 bkXP
bkXbnkXPbknkRKK |2,2
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 25 of 54 ECE 3800
But now, the conditional probability is based on an even number of state transitions in k steps. This is a summation of the Binomial probability (with an adjustment for “change” and “no change”.
lnll pp
l
np
1
n
evenl
lnl ppl
nbkXbnkXP 1|
Using the textbook trick
Note, I haven’t seen this result before nn
l
lnl pppl
n121
0
1122
11|
nn
evenl
lnl pppl
nbkXbnkXP
And the result becomes
1122
1
2
2
nKK p
bnR
1124
2
nKK p
bnR
Note that the autocovariance is
4
1124
222 b
pb
nRnK nXKKKK
nKK p
bnK 12
4
2
We can tackle the more complex problem of a not equal to zero now … we needed Ae and Ao.
kbnkaPkbPabkankbPkaPba
kankaPkaPaakbnkbPkbPbbknkRKK ;|;;;|;;
;|;;;|;;,
OO
eeKK AkbPabAkaPba
AkaPaaAkbPbbknkR
;;
;;,
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 26 of 54 ECE 3800
For
nE pA 121
2
1
nO pA 121
2
1
OEKK AbaA
abknkR
22,
22
nn
KK pba
pab
knkR 1212
12144
,22
nKK p
babanR 12
44
22
Note: the simulation has “flipped p and q so that
nnKK p
babaq
babanR 21
4412
44
2222
p=0.75, a=-1, b=1
p=0.25, a=-1, b=1
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 27 of 54 ECE 3800
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 28 of 54 ECE 3800
Note: the following examples are going to use continuous time instead of discrete time. These examples could be made discrete, but the concept remains the same. In the textbook, the continuous time derivation is in Chapter 9.
Example: tfAtx 2sin for a uniformly distributed random variable 2,0
212121 2sin2sin, tfAtfAEtXtXEttRXX
22cos2cos2
1, 2121
22121 ttfttfEAtXtXEttRXX
22cos2
2cos2
, 21
2
21
2
2121 ttfEA
ttfA
tXtXEttRXX
Of note is that the phase need only be uniformly distributed over 0 to π in the previous step!
21
2
2121 2cos2
, ttfA
tXtXEttRXX
for 21 tt
fA
RXX 2cos2
2
but
fA
RR XXXX 2cos2
2
Assuming a uniformly distributed random phase “simplifies the problem” …
Also of note, if the amplitude is an independent random variable, then
fAE
RXX 2cos2
2
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 29 of 54 ECE 3800
Example:
T
ttrectBtx 0 for B =+/-A with probability p and (1-p) and t0 a uniformly
distributed random variable
2,
20
TTt . Assume B and t0 are independent.
T
ttrectB
T
ttrectBEtXtXEttRXX
02012121 ,
T
ttrect
T
ttrectBEtXtXEttRXX
020122121 ,
As the RV are independent
T
ttrect
T
ttrectEBEtXtXEttRXX
020122121 ,
T
ttrect
T
ttrectEpApAttRXX
02012221 1,
2
2
002012
21
1,
T
TXX dt
TT
ttrect
T
ttrectAttR
For 21 0 tandt
2
2
002 1
1,0
T
TXX dt
TT
trectAR
The integral can be recognized as being a triangle, extending from –T to T and zero everywhere else.
TtriARXX
2
T
TTT
A
TTT
A
T
RXX
,0
0,1
0,1
,0
2
2
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 30 of 54 ECE 3800
Properties of Autocorrelation Functions (with WSS property)
1) 220 XXERXX or 20 txXX
the mean squared value of the random process can be obtained by observing the zeroth lag of the autocorrelation function.
2) XXXX RR
The autocorrelation function is an even function in time. Only positive (or negative) needs to be computed for an ergodic, WSS random process. (Conjugate symmetry if complex)
3) 0XXXX RR
The autocorrelation function is a maximum at 0. For periodic functions, other values may equal the zeroth lag, but never be larger.
4) If X has a DC component, then Rxx has a constant factor.
tNXtX
NNXX RXR 2
Note that the mean value can be computed from the autocorrelation function constants!
5) If X has a periodic component, then Rxx will also have a periodic component of the same period.
Think of:
20,cos twAtX
where A and w are known constants and theta is a uniform random variable.
wA
tXtXERXX cos2
2
5b) For signals that are the sum of independent random variable, the autocorrelation is the sum of the individual autocorrelation functions.
tYtXtW
YXYYXXWW RRR 2
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 31 of 54 ECE 3800
For non-zero mean functions, (let w, x, y be zero mean and W, X, Y have a mean)
YXYYXXWW RRR 2
YXYyyXxxWwwWW RRRR 2222
222 2 YYXXyyxxWwwWW RRRR
22YXyyxxWwwWW RRRR
Then we have
22YXW
yyxxww RRR
6) If X is ergodic and zero mean and has no periodic component, then
0lim
XXR
No good proof provided.
The logical argument is that eventually new samples of a non-deterministic random process will become uncorrelated. Once the samples are uncorrelated, the autocorrelation goes to zero.
7) Autocorrelation functions can not have an arbitrary shape. One way of specifying shapes permissible is in terms of the Fourier transform of the autocorrelation function. That is, if
dtjwtRR XXXX exp
then the restriction states that
wallforRXX 0
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 32 of 54 ECE 3800
Additional concepts that are useful … dealing with constant:
tNatX
tNatNaEtXtXERXX
NNXX RatNtNEaR 22
tYatX
tYatYaERXX
tYtYtYatYaaERXX2
tYtYEtYEatYEaaRXX2
YYXX RtYEaaR 22
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 33 of 54 ECE 3800
8.2 Basic Principles of Discrete-Time Linear Systems
We get to do convolutions some more … in the discrete time domain!
Note: if you are in ECE 3710, this should be normal; otherwise, ECE 3100 probably talked about linear systems being a convolution.
For a “causal” discrete finite impulse response linear system we will have ….
mxmnhknxkhnyn
mk
0
For a “non-causal” discrete linear system we will have ….
mxmnhknxkhnymk
Foralinearsystem,superpositionapplies
nyny
knxkhaknxkha
knxakhknxakh
knxaknxakhny
kk
kk
k
21
20
210
1
220
110
22110
For a filter with poles and zeros …. the filter may be autoregressive as well!
knykaknxkbnykk
10
This is called a linear constant coefficient difference equation (LCCDE) in the text.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 34 of 54 ECE 3800
Solvingdifferenceequations
Example 8.2-1 nunynyny 0.1272.017.1
Need assumptions: y(-1)=0 and y(-2)=0
First solve the recursive part of the equation (homogeneous equation) then solve for the input and combine the two solutions (superposition works).
272.017.1 nynyny hhh
A prototype solution is: nh rAny
Plugging it in … 0272.017.1 nynyny hhh
072.07.1 21 nnn rArArA
072.07.12 rr
Solve for the roots
a
cab
a
br
2
4
2
2
2,1
2
72.047.1
2
7.12
2,1
r
9.0,8.005.085.02
88.289.285.02,1
r
Therefore, since the roots are less than one, the equation will be stable and … nn
h AAny 9.08.0 21
Looking at the particular solution …
The input is a unit step function, x(n) = 1 for n>=0. There should be a steady state solution as n goes to infinity. If so, y(n)=y(n-1)-y(n-2) … for very large n
172.07.1 yyy
Therefore, 172.07.11 y
5002.01 y
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 35 of 54 ECE 3800
Then, 509.08.0 21 nn
h AAynyny
Now for the specifics, determine the result for y(1) and y(2) and then solve for the two remaining unknowns in A. Given y(-1)=0 and y(-2)=0 …
0.10.1272.017.10 yyy
7.20.17.10.1172.007.11 yyy
Using the prototype solution
1509.08.00 02
01 AAy
7.2509.08.01 12
11 AAy
You like doing a matrix solution?
8.47
0.49
9.08.0
11
2
1
A
A
0.81
0.32
8.47
0.49
8.09.0
18.0
19.0
2
1
A
A
and nforny nn 0,509.0818.032
Lineartimeinvariantandlinearshiftinvariant
nallforknxLkny ,
A time offset in the input will not change the response at the output! This is key to the convolution theory!
SystemImpulseresponse
The response to a unit impulse is the impulse response
nhnLny
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 36 of 54 ECE 3800
ThediscreteFourierTransformexists(ifx(n)isbounded).Dr.Bazuinmayalsocallthisthediscrete‐time,continuous‐frequencyFourierTransform(somethingfromECE4550)
n
nwjnxnxwX exp , for w
and X(w) periodic in 2 pi outside the range (it repeats in the frequency domain)
The inverse transform is defined as
dwnwjwXwXnx exp
2
11
Theorem8.2‐1:Convolutioninthetimedomainisequivalenttomultiplicationinthefrequencydomain
nn
nwjnhnxnwjnywY expexp
n m
nwjmxmnhwY exp
Switch the order of summation and maintain terms …
mxnwjmnhwYm n
exp
mwjmxmnwjmnhwYm n
expexp
The inside summation is a discrete FT
mm
mwjmxwHmwjmxwHwY expexp
wXwHwY
Thez‐Transformexists(withtheappropriateregionofconvergence(ROC)).
n
nznxzX
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 37 of 54 ECE 3800
8.3 Random Sequences and Linear Systems 477
For a “non-causal” discrete linear system we will have …. (FIR filter)
mxmnhEknxkhEnyEmk
mxEmnhknxEkhnyEmk
If WSS
Xm
Xk
Y mnhkhnyE
k
XY khnyE
The mean times the coherent gain of the filter.
For a filter with poles and zeros …. the filter may be autoregressive as well!
knykaEknxkbEnyEkk 10
knyEkaknxEkbnyEkk
10
If WSS
Yk
Xk
Y kakbnyE
10
01
1k
Xk
Y kbka
1
0
1k
kXY
ka
kb
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 38 of 54 ECE 3800
If you are familiar with the z-transform
n
nznxzX
1
0
1k
k
k
k
zka
zkb
zX
zYzH
Therefore
1
11
0
zHka
kb
X
k
kXY
If in the Fourier Domain
0
11
0
wHka
kb
X
k
kXY
For an analog system …
For he Laplace transform, the “final value theorem” or steady state at infinite time is sXsx
s
0lim
But for a unit step response input to a transfer function,
s
tu1
Therefore, for a unit step “DC level” input, the steady state output is
01
limlim0
Hs
sHstuthyst
For sB
sAsH
It is just the sum of the numerator coefficients divided by the sum of the denominator coefficients!
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 39 of 54 ECE 3800
Auto- and Cross-Correlation
For a “causal” discrete finite impulse response linear system we will have … (impulse response based)
mxmnhknxkhnyn
mk
0
And performing a cross-correlation (assuming real R.V. and processing)
knxkhnxEnynxEk
20
121
knxnxkhEnynxEk
210
21
knxnxEkhnynxEk
21
021
knnRkhnynxE XXk
21
021 ,
For x(n) WSS
nkmnRkhmRmnynxE XXk
XY
0
kmRkhmRmnynxE XXk
XY
0
mRmhmRmnynxE XXXY
What about the other way … YX instead of XY
2
0121 nxknxkhEnxnyE
k
210
21 nxknxEkhnxnyEk
210
21 , nknRkhnxnyE XXk
For x(n) WSS … see the next page
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 40 of 54 ECE 3800
For x(n) WSS
knmnRkhmRmnxnyE XXk
YX
0
kmRkhmRmnxnyE XXk
YX
0
Perform a change of variable for k to “-l” (assuming h(t) is real, see text for complex
lmRlhmRmnxnyE XXl
YX
0
Therefore mRmhmRmnxnyE XXYX
Whatabouttheauto‐correlationofy(n)?
And performing an auto-correlation (assuming real R.V. and processing)
22
0211
0121
21
knxkhknxkhEnynyEkk
0 022112121
1 2k k
knxknxkhkhEnynyE
0 0
22112121
1 2k k
knxknxEkhkhnynyE
0 0
22112121
1 2
,k k
XX knknRkhkhnynyE
For x(n) WSS
0 0
1221
1 2k kXXYY knkmnRkhkhmRmnynyE
0 0
2121
1 2k kXXYY kkmRkhkhmRmnynyE
The output autocorrelation can also be defined in terms of the cross-correlation as
0
11
1kXYYY kmRkhmRmnynyE
mhmRmRmhmRmnynyE XYXYYY
The cross-correlation can be used to determine the output auto-correlation!
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 41 of 54 ECE 3800
Continue in this concept, the cross correlation is also a convolution. Therefore,
mhmRmRmnynyE XYYY
mhmhmRmRmnynyE XXYY
If h(n) is complex, the term in h(-n) must be a conjugate.
Summary: For x(n) WSS and a real filter
mRmhmRmnynxE XXXY
mRmhmRmnxnyE XXYX
mhmhmRmRmnynyE XXYY
TheMeanSquareValueataSystemOutput
0 0
21212
1 2
00k k
XXYY kkRkhkhRnyE
0 0
21212
1 2
0k k
XXYY kkRkhkhRnyE
0
2 0k
XXYY kRkhkhRnyE
or
0
2 0k
XYYY kRkhRnyE
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 42 of 54 ECE 3800
Example:WhiteNoiseInputstoacausalfilter
Let nN
nRXX 2
0
0 0
21212
1 2
0k k
XXYY kkRkhkhRnyE
0 0
210
212
1 22
0k k
YY kkN
khkhRnyE
0
1102
12
0k
YY khkhN
RnyE
0
202
20
kYY kh
NRnyE
For a white noise process, the mean squared (or 2nd moment) is proportional to the filter power.
Typically, there are similar derivations for sampled systems and continuous systems. .
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 43 of 54 ECE 3800
The power spectral density output of linear systems 489
The discrete Power Spectral Density is defined as:
n
XXXX nwjnRwS exp
The inverse transform is defined as
dwnwjwSwSnR XXXXXX exp
2
11
Properties:
1. Sxx(w) is purely real as Rxx(n) is conjugate symmetric
2. If X(n) is a real-valued WSS process, then Sxx(w) is an even function, as Rxx(n) is real and even.
3. Sxx(w)>= 0 for all w.
4. Rxx(m)=0 for all m>N for some finite integer. This is the condition for the Fourier transform to exist … finite energy.
Cross‐SpectralDensity
Since we have already shown the convolution formula, we can progress to the cross-spectral density functions
n
XYXY nwjnRwS exp
n
XXXY nwjnRnhwS exp
Then
wHwSwS XXXY
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 44 of 54 ECE 3800
And for the other cross spectral density
n
YXYX nwjnRwS exp
n
XXXY nwjmRmhwS exp
Then for a real filter
*wHwSwHwSwS XXXXXY
Theoutputpowerspectraldensitybecomes
n
YYYY nwjnRwS exp
n
XXYY nwjmhmhmRwS exp*
For all systems
2* wHwSwHwHwSwS XXXXYY
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 45 of 54 ECE 3800
Note: The author and Dr. Bazuin differ in the ordering of the cross-spectral densities.
There is a philosophy to the notations …
Do you define it as
mRmnynyE YY *
12*
21 nnRnynyE YY
or as
mRnymnyE YY * where mRmnynyE YY *
1221*
21 nnRnnRnynyE YYYY
Dr. Bazuin prefers the former while the authors prefer the later!
For real R.V. or functions, it really doesn’t matter for auto-correlation mRmR YYYY
But, for cross-correlation functions you must be consistent using one way or the other and not mixing them! Although again for real functions mRmR YXXY
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 46 of 54 ECE 3800
Examples using the Autocorrelation and PSD
Example8.4‐2EdgeDetector(1storderdifferenceorderivative)
1 nXnXnY
The filter that generates this can be described as
1 nnnh
If the output autocorrelation can be described as
mhmhmRmRmnynyE XXYY
We can form
mgmRmRmnynyE XXYY
where mhmhmg
For this problem we can compute
k
kmhkhmg
k
kmkmkkmg 11
There will be four terms to consider:
k
kmkkmkkmkkmkmg 1111
Based on arbitrary time differences … these reduce to
11211
mmmmmmmmgk
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 47 of 54 ECE 3800
Then
k
XXYY kgkmRmR
1
1
112k
XXYY kmkkmRmR
112 mRmRmRmR XXXXXXYY
If the input autocorrealtion is defined as
mXX mR
Figure 8.3-2 Input correlation function for edge detector with a = 0.7.
Then
112 mmmYY mR
What about the power spectral density?
The first term is easy, but the others are tougher … ie is easier to calculate as the product of the input PSD and the filter magnitude squared.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 48 of 54 ECE 3800
Power spectral density approach
2* wHwSwHwHwSwS XXXXYY
From the filter
nn
nwjnnnwjnhwH exp1exp
wjwH exp1
wjwjwHwHwH exp1exp1*2
wwjwjwH cos221expexp12
From the input (the easy term from before)
n
XXXX nwjnRwS exp
n
nXX nwjwS exp
10
expexpn
n
n
nXX nwjnwjwS
00
expexp1n
n
n
nXX wjwjwS
wjwjwS XX
exp1
1
exp1
11
2expexp1
exp1exp11
wjwj
wjwjwS XX
2
2
2 cos21
cos21cos22
cos21
cos221
w
ww
w
wwS XX
2
2
cos21
1
w
wS XX
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 49 of 54 ECE 3800
And finally
2* wHwSwHwHwSwS XXXXYY
ww
wSYY cos22cos21
12
2
2
2
cos21
cos112
w
wwSYY
So is this better or worse than doing a direct Fourier transform?
Frequency response notes:
1
1
1
11
21
1
cos21
10
22
2
2
2
wS XX
0cos2202 wH
0
1
1112
cos21
cos1120
2
2
w
wSYY
and
1
1
1
11
21
1
cos21
122
2
2
2
wS XX
4cos222 wH
1
14
1
1112
cos21
cos112
2
2
w
wSYY
see Example_SW_8_4_2.m
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 50 of 54 ECE 3800
SynthesisofRandomSequencesandDiscrete‐TimeSimulations
We can generate a transfer function to provide a random sequence with a specified PSD or correlation function. Staring with a digital filter
knykaknxkbnykk
10
The Fourier transform is
wB
wA
wX
wYwH
where
0
expn
nwjnbwB and
1
exp1n
nwjnawA
The signal input to the filter is white noise, producing a constant magnitude frequency response. Therefore,
20*0
22wH
NwHwH
NwSYY
For real causal coefficients, this is also equivalent to
wHwHN
wSYY 2
0
Or using z-transform notation for wjz exp then wjz exp1 this can be written as
10
2 zHzH
NzSYY
In the z-domain, the unit circle is a key component where this implies that there is a mirror image about the unit circle for poles and zero in the z-domain. As an added point of interest, minimum phase, stable filters will have all their poles and zeros inside the unit circle. The mirror image elements form the “inverse filter”.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 51 of 54 ECE 3800
Example8.4‐5Filtergeneration
If a desired psd can be stated as
2
2
cos21
wwS N
XX
For wjwjw expexpcos2 and wjwj expexp1
The equivalent z-transform representation is
zzw 1cos2 and zz 11
Then
zzzzzS N
XX
11
2
1
zHzHzzzz
zS NNNXX
121
21
2
1
1
1
1
11
1
The desired filter is then
z
zH
1
1
The inverse transform becomes
nunh n
or
1 nynxny
Note: this should look similar to Example 8.4-6 ….
If you are interested in Decimation 496 Interpolation 497 the basis for this will be shown in ECE 4550 and you could see me for a multirate signal processing (graduate) course ….
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 52 of 54 ECE 3800
Discrete Implementation – Notes on the Discrete Fourier Transform
The discrete Fourier Transform
1
0
WN
n
nkND nxkX
1
0
W1
ˆN
n
Dnk
N kXN
kx
where
N
nkjnkN
2expW
Equivalence of the continuous and discrete transforms …
(1) Discretization of the input tnxnx for sampleft 1
(2) Spectral bins “actual center freq” kXfkXtN
kX DCC
where k=0, +/-1, … +/- N/2, and Nf
f sample
DFTs get it right at the frequency sample points … based on the applied or actual window used.
(3) Spectral bin concept kXdftN
kX
f D
Nf
Nf
C
sample
sample
2
2
1
This is a concept; reality requires that the discrete point consist of the complete “windowed” signal response. (see the Matlab windows). This can be thought of as a piecewise linear estimate of the bin energy.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 53 of 54 ECE 3800
DFTSymmetryforrealnumbers
kXkNX
nxnxkNX
nxkX
N
n
nkN
nNN
N
n
kNnN
N
n
nkN
*
1
0
1
0
1
0
WWW
W
kYkNY
nyikNY
nyikNY
nyinyikNY
nyikY
N
n
nkN
N
n
nkN
N
n
N
n
nkN
nNN
kNnN
N
n
nkN
*
1
0
*
1
0
1
0
1
0
1
0
W
W
WWW
W
Notice the spectral symmetries involved. Purely real inputs are conjugate symmetric about N, while imaginary inputs are conjugate anti-symmetric about N.
For real input, bins 0 and N/2 are special, while
12
12
22
11
*
*
*
NX
NX
XNX
XNX
Note that k=0 to N/2-1 represent the positive spectrum, while N/2+1 to N-1 represent the negative spectrum.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 54 of 54 ECE 3800
DFTSymmetryforevensymmetricsequences
1:0,22 NnnNxnx
11
2
321
220
NxNx
NxNx
Nxx
Nxx
Note that x(N-1) is singular … it is symmetric to itself
22
022W
N
n
nkN nxkX
22
22122
2
022 W1WW
N
Nn
nkN
kNN
N
n
nkN nxNxnxkX
2
0
22222
2
022 22W1WW
N
n
knNN
kN
n
nkN nNxNxnxkX
2
022
22222
2
022 22WW1WW
N
n
nkN
kNN
kN
n
nkN nNxNxnxkX
2
0222
2
022 W1WW
N
n
nkN
kN
n
nkN nxNxnxkX
1WWW 2
2
02222
NxnxkX k
N
n
nkN
nkN
1122
2cos22
0
NxnxN
knkX k
N
n
Therefore, X(k) is purely real … no phase components as we would require.
Since x(n) is real, we must also have … “conjugate symmetry” but there is no imaginary part!
kNXkX 2
Therefore, only k=0 to N-1 must be computed!
Top Related