Basic Definitions
and
The Spectral Estimation Problem
Lecture 1
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–1by P. Stoica and R. Moses, Prentice Hall, 1997
Informal Definition of Spectral Estimation
Given: A finite record of a signal.
Determine: The distribution of signal power over
frequency.
t
signal
t=1, 2, . . . ω ω+Δωω
π
spectral density
� � (angular) frequency in radians/(sampling interval)
f � ���� = frequency in cycles/(sampling interval)
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–2by P. Stoica and R. Moses, Prentice Hall, 1997
Applications
Temporal Spectral Analysis
� Vibration monitoring and fault detection
� Hidden periodicity finding
� Speech processing and audio devices
� Medical diagnosis
� Seismology and ground movement study
� Control systems design
� Radar, Sonar
Spatial Spectral Analysis
� Source location using sensor arrays
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–3by P. Stoica and R. Moses, Prentice Hall, 1997
Deterministic Signals
fy�t�g�t��� � discrete-time deterministic data
sequence
If:
�Xt���
jy�t�j� ��
Then: Y ��� =
�Xt���
y�t�e�i�t
exists and is called the Discrete-Time Fourier Transform(DTFT)
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–4by P. Stoica and R. Moses, Prentice Hall, 1997
Energy Spectral Density
Parseval's Equality:
�Xt���
jy�t�j� =
���
Z ���
S���d�
where
S���
�� jY ���j�
� Energy Spectral Density
We can write
S��� �
�Xk���
��k�e�i�k
where
��k� �
�Xt���
y�t�y��t� k�Lecture notes to accompany Introduction to Spectral Analysis Slide L1–5by P. Stoica and R. Moses, Prentice Hall, 1997
Random Signals
Random Signal
t
random signal probabilistic statements about future variations
current observation time
Here:
�Xt���
jy�t�j� ��
But: Enjy�t�j�
o��
E f�g = Expectation over the ensemble of realizations
Enjy�t�j�
o
= Average power in y�t�
PSD = (Average) power spectral density
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–6by P. Stoica and R. Moses, Prentice Hall, 1997
First Definition of PSD
���� =
�Xk���
r�k�e�i�k
where r�k� is the autocovariance sequence (ACS)
r�k� � E fy�t�y��t� k�g
r�k� � r���k�� r��� � jr�k�j
Note that
r�k� �
���
Z ���
����ei�kd� (Inverse DTFT)
Interpretation:
r��� � Enjy�t�j�
o�
���
Z ���
����d�
so
����d� � infinitesimal signal power in the band
� � d��
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–7by P. Stoica and R. Moses, Prentice Hall, 1997
Second Definition of PSD
���� = limN��
E���
���
N������NX
t��y�t�e�i�t
����������
�
Note that���� � limN��
E�
NjYN���j�
�
where
YN��� �
NXt��
y�t�e�i�t
is the finite DTFT of fy�t�g.
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–8by P. Stoica and R. Moses, Prentice Hall, 1997
Properties of the PSD
P1: ���� � ������� for all �.
Thus, we can restrict attention to
� � ���� � � f � ���������
P2: ���� � �
P3: If y�t� is real,
Then: ���� = �����
Otherwise: ���� � �����
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–9by P. Stoica and R. Moses, Prentice Hall, 1997
Transfer of PSD Through Linear Systems
System Function: H�q� �
�Xk��
hkq�k
where q�� � unit delay operator: q��y�t� � y�t� ��
e�t��e���
y�t�
�y��� � jH���j��e���
H�q� ��
Then
y�t� =
�Xk��
hk e�t� k�
H��� =
�Xk��
hk e�i�k
�y��� = jH���j��e���
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–10by P. Stoica and R. Moses, Prentice Hall, 1997
The Spectral Estimation Problem
The Problem:
From a sample fy���� � � � � y�N�g
Find an estimate of ����: f����� � � ���� �g
Two Main Approaches :
� Nonparametric:
– Derived from the PSD definitions.
� Parametric:
– Assumes a parameterized functional form of the
PSD
Lecture notes to accompany Introduction to Spectral Analysis Slide L1–11by P. Stoica and R. Moses, Prentice Hall, 1997
Periodogram
and
Correlogram
Methods
Lecture 2
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–1by P. Stoica and R. Moses, Prentice Hall, 1997
Periodogram
Recall 2nd definition of ����:
���� � limN��
E���
���
N������NX
t��y�t�e�i�t
����������
�
Given : fy�t�gNt��
Drop “ limN��
” and “E f�g” to get
�p��� =
�N
������NX
t��y�t�e�i�t
�������
� Natural estimator
� Used by Schuster (�1900) to determine “hidden
periodicities” (hence the name).
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–2by P. Stoica and R. Moses, Prentice Hall, 1997
Correlogram
Recall 1st definition of ����:
���� �
�Xk���
r�k�e�i�k
Truncate the “
P” and replace “r�k�” by “r�k�”:
�c��� =
N��Xk���N���
r�k�e�i�k
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–3by P. Stoica and R. Moses, Prentice Hall, 1997
Covariance Estimators(or Sample Covariances)
Standard unbiased estimate:
r�k� �
�N � k
NXt�k��
y�t�y��t� k�� k � �
Standard biased estimate:
r�k� �
�N
NXt�k��
y�t�y��t� k�� k � �
For both estimators:
r�k� � r���k�� k � �
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–4by P. Stoica and R. Moses, Prentice Hall, 1997
Relationship Between ��p��� and ��c���
If: the biased ACS estimator r�k� is used in �c���,
Then:
�p��� �
�N
������NX
t��y�t�e�i�t
�������
�
N��Xk���N���
r�k�e�i�k
� �c���
�p��� � �c���
Consequence:Both �p��� and �c��� can be analyzed simultaneously.
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–5by P. Stoica and R. Moses, Prentice Hall, 1997
Statistical Performance of ��p��� and ��c���
Summary:
� Both are asymptotically (for large N ) unbiased:
En�p���
o� ���� as N ��
� Both have “large” variance, even for large N .
Thus, �p��� and �c��� have poor performance.
Intuitive explanation:
� r�k�� r�k� may be large for large jkj
� Even if the errors fr�k�� r�k�gN��
jkj��
are small,
there are “so many” that when summed in
��p���� ����, the PSD error is large.
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–6by P. Stoica and R. Moses, Prentice Hall, 1997
Bias Analysis of the Periodogram
En�p���
o� E
n�c���
o�
N��Xk���N���
E fr�k�g e�i�k
�
N��Xk���N���
���jkj
N
r�k�e�i�k
�
�Xk���
wB�k�r�k�e�i�k
wB�k� �
���
���jkjN
�� jkj N � �
�� jkj � N
� Bartlett, or triangular, window
Thus,
En�p���
o�
���
Z ���
���WB�� � � d
Ideally: WB��� � Dirac impulse ���.
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–7by P. Stoica and R. Moses, Prentice Hall, 1997
Bartlett Window WB���
WB��� �
�N
�sin��N���
sin�������
WB����WB���, for N � ��
−3 −2 −1 0 1 2 3−60
−50
−40
−30
−20
−10
0
dB
ANGULAR FREQUENCY
Main lobe 3dB width� ��N .
For “small” N , WB��� may differ quite a bit from ���.
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–8by P. Stoica and R. Moses, Prentice Hall, 1997
Smearing and Leakage
Main Lobe Width: smearing or smoothing
Details in ���� separated in f by less than ��N are not
resolvable.φ(ω)
ω<1/Ν ω
φ(ω)^smearing
Thus: Periodogram resolution limit = ��N .
Sidelobe Level: leakage
φ(ω)≅δ(ω)
ω ω
φ(ω)≅W (ω)^
leakage
B
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–9by P. Stoica and R. Moses, Prentice Hall, 1997
Periodogram Bias Properties
Summary of Periodogram Bias Properties:
� For “small” N , severe bias
� As N ��, WB���� ���,
so ���� is asymptotically unbiased.
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–10by P. Stoica and R. Moses, Prentice Hall, 1997
Periodogram Variance
As N ��
Enh�p����� �����
i h�p����� �����
io
�
�������� �� � ��
�� �� � ��
� Inconsistent estimate
� Erratic behavior
ω
φ(ω)^
1 st. dev = φ(ω) too +-
asymptotic mean = φ(ω)
φ(ω)^
Resolvability properties depend on both bias and variance.
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–11by P. Stoica and R. Moses, Prentice Hall, 1997
Discrete Fourier Transform (DFT)
Finite DTFT: YN��� �
NXt��
y�t�e�i�t
Let � � ��N k and W � e�i
��N .
Then YN���N k� is the Discrete Fourier Transform (DFT):
Y �k� �
NXt��
y�t�W tk� k � �� � � � � N � �
Direct computation of fY �k�gN��k�� from fy�t�gNt��:
O�N�� flops
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–12by P. Stoica and R. Moses, Prentice Hall, 1997
Radix–2 Fast Fourier Transform (FFT)
Assume: N � �m
Y �k� �
N��Xt��
y�t�W tk �
NXt�N����
y�t�W tk
�
N��Xt��
�y�t� � y�t�N���WNk� W tk
with WNk� �
��� for even k
��� for odd k
Let �N � N�� and �W �W� � e�i��� �N .
For k � ���� � � � � � N � ��� �p:
Y ��p� �
�NXt��
�y�t� � y�t� �N� �W tp
For k � ������ � � � � N � � � �p��:
Y ��p��� �
�NXt��
f�y�t�� y�t� �N�W tg �W tp
Each is a �N � N��-point DFT computation.
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–13by P. Stoica and R. Moses, Prentice Hall, 1997
FFT Computation Count
Let ck � number of flops for N � �k point FFT.
Then
ck �
�k�
� �ck��
ck �
k�k�
Thus,
ck �
��N log�N
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–14by P. Stoica and R. Moses, Prentice Hall, 1997
Zero Padding
Append the given data by zeros prior to computing DFT
(or FFT):
fy���� � � � � y�N���� � � � �� �z �
N
g
Goals:
� Apply a radix-2 FFT (so N � power of 2)
� Finer sampling of ����:��
Nk
�N��k��
�
��
Nk
�N��k��
ω
φ(ω)^continuous curve
sampled, N=8
Lecture notes to accompany Introduction to Spectral Analysis Slide L2–15by P. Stoica and R. Moses, Prentice Hall, 1997
Improved Periodogram-Based
Methods
Lecture 3
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–1by P. Stoica and R. Moses, Prentice Hall, 1997
Blackman-Tukey Method
Basic Idea: Weighted correlogram, with small weight
applied to covariances r�k� with “large” jkj.
�BT ��� �
M��Xk���M���
w�k�r�k�e�i�k
fw�k�g � Lag Window
1
-M M
w(k)
k
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–2by P. Stoica and R. Moses, Prentice Hall, 1997
Blackman-Tukey Method, con't
�BT ��� �
���
Z ���
�p��W�� � �d
W��� � DTFTfw�k�g
� Spectral Window
Conclusion: �BT ��� � “locally” smoothed periodogram
Effect:
� Variance decreases substantially
� Bias increases slightly
By proper choice of M :
MSE � var � bias� � � as N ��Lecture notes to accompany Introduction to Spectral Analysis Slide L3–3by P. Stoica and R. Moses, Prentice Hall, 1997
Window Design Considerations
Nonnegativeness:
�BT ��� �
���
Z ���
�p��� �z ���
W�� � �d
If W��� � � �� w�k� is a psd sequence)
Then: �BT ��� � � (which is desirable)
Time-Bandwidth ProductNe �
M��Xk���M���
w�k�
w���
� equiv time width
�e �
���
Z ���
W���d�
W���
� equiv bandwidth
Ne �e � �
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–4by P. Stoica and R. Moses, Prentice Hall, 1997
Window Design, con't
� �e � ��Ne � ����M�
is the BT resolution threshold.
� As M increases, bias decreases and variance
increases.
Choose M as a tradeoff between variance and
bias.
� Once M is given, Ne (and hence �e) is essentially
fixed.
Choose window shape to compromise between
smearing (main lobe width) and leakage (sidelobe
level).
The energy in the main lobe and in the sidelobes
cannot be reduced simultaneously, once M is given.
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–5by P. Stoica and R. Moses, Prentice Hall, 1997
Window Examples
Triangular Window, M � ��
−3 −2 −1 0 1 2 3−60
−50
−40
−30
−20
−10
0
dB
ANGULAR FREQUENCY
Rectangular Window, M � ��
−3 −2 −1 0 1 2 3−60
−50
−40
−30
−20
−10
0
dB
ANGULAR FREQUENCY
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–6by P. Stoica and R. Moses, Prentice Hall, 1997
Bartlett Method
Basic Idea:1 Μ 2Μ Ν. . .
average. . .
φ (ω)1̂
φ (ω)2̂
φ (ω)L̂
φ (ω)B̂
Mathematically:
yj�t� � y��j � ��M � t� t � �� � � � �M
� the jth subsequence
�j � �� � � � � L�� �N�M �
�j��� �
�M
������MX
t��yj�t�e�i�t
�������
�B��� �
�L
LXj��
�j���
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–7by P. Stoica and R. Moses, Prentice Hall, 1997
Comparison of Bartlettand Blackman-Tukey Estimates
�B��� �
�L
LXj��
�����
M��Xk���M���
rj�k�e�i�k
����
�
M��Xk���M���
����L
LXj��
rj�k���
e�i�k
�
M��Xk���M���
r�k�e�i�k
Thus:�B��� � �BT ��� with a rectangular
lag window wR�k�
Since �B��� implicitly uses fwR�k�g, the Bartlettmethod has
� High resolution (little smearing)
� Large leakage and relatively large variance
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–8by P. Stoica and R. Moses, Prentice Hall, 1997
Welch Method
Similar to Bartlett method, but
� allow overlap of subsequences (gives moresubsequences, and thus “better” averaging)
� use data window for each periodogram; givesmainlobe-sidelobe tradeoff capability
1 2 N
subseq#1
. . .
subseq#2
subseq#S
Let S �# of subsequences of length M .(Overlapping means S � �N�M “betteraveraging”.)
Additional flexibility:
The data in each subsequence are weighted by a temporalwindow
Welch is approximately equal to �BT ��� with anon-rectangular lag window.
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–9by P. Stoica and R. Moses, Prentice Hall, 1997
Daniell Method
By a previous result, for N � �,
f�p��j�g are (nearly) uncorrelated random variables for
�j �
��N
j�N��
j��
Idea: “Local averaging” of ��J ��� samples in the
frequency domain should reduce the variance by about��J ���.
�D��k� �
�
�J ��
k�JXj�k�J
�p ��j�
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–10by P. Stoica and R. Moses, Prentice Hall, 1997
Daniell Method, con't
As J increases:
� Bias increases (more smoothing)
� Variance decreases (more averaging)
Let � � �J�N . Then, for N � �,
�D��� �
����
Z �����
�p���d�
Hence: �D��� � �BT ��� with a rectangular spectralwindow.
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–11by P. Stoica and R. Moses, Prentice Hall, 1997
Summary of Periodogram Methods
� Unwindowed periodogram– reasonable bias– unacceptable variance
� Modified periodograms– Attempt to reduce the variance at the expense of (slightly)
increasing the bias.
� BT periodogram– Local smoothing/averaging of �p��� by a suitably selected
spectral window.
– Implemented by truncating and weighting r�k� using a lagwindow in �c���
� Bartlett, Welch periodograms– Approximate interpretation: �BT ��� with a suitable lag
window (rectangular for Bartlett; more general for Welch).
– Implemented by averaging subsample periodograms.
� Daniell Periodogram– Approximate interpretation: �BT ��� with a rectangular
spectral window.
– Implemented by local averaging of periodogram values.
Lecture notes to accompany Introduction to Spectral Analysis Slide L3–12by P. Stoica and R. Moses, Prentice Hall, 1997
Parametric Methods
for
Rational Spectra
Lecture 4
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–1by P. Stoica and R. Moses, Prentice Hall, 1997
Basic Idea of Parametric Spectral Estimation
Observed Data
Assumed functional
form of φ(ω,θ)
Estimate parametersin φ(ω,θ)
Estimate PSD
φ(ω) =φ(ω,θ)^ ^θ̂
possibly revise assumption on φ(ω)
Rational Spectra���� �
Pjkj�m ke
�i�kPjkj�n �ke�i�k
���� is a rational function in e�i�.
By Weierstrass theorem, ���� can approximate arbitrarilywell any continuous PSD, provided m and n are chosensufficiently large.
Note, however:
� choice of m and n is not simple
� some PSDs are not continuous
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–2by P. Stoica and R. Moses, Prentice Hall, 1997
AR, MA, and ARMA Models
By Spectral Factorization theorem, a rational ���� can be
factored as
���� �
�����B���A���������
��
A�z� � �� a�z��� � � �� anz�n
B�z� � �� b�z��� � � �� bmz�m
and, e.g., A��� � A�z�jz�ei�
Signal Modeling Interpretation:
e�t�
�e��� � ��
white noise
y�t�
�y��� �����
B���
A�������
���
B�q�
A�q�
�ltered white noise
��
ARMA: A�q�y�t� � B�q�e�t�
AR: A�q�y�t� � e�t�
MA: y�t� � B�q�e�t�
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–3by P. Stoica and R. Moses, Prentice Hall, 1997
ARMA Covariance Structure
ARMA signal model:
y�t��
nXi��
aiy�t� i� �
mXj��
bje�t�j�� �b� � ��
Multiply by y��t� k� and take E f�g to give:
r�k� �
nXi��
air�k � i� �
mXj��
bjE fe�t� j�y��t� k�g
� ��mX
j��bjh�j�k
� � for k � m
where H�q� � B�q�
A�q��
�Xk��
hkq�k� �h� � ��
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–4by P. Stoica and R. Moses, Prentice Hall, 1997
AR Signals: Yule-Walker Equations
AR: m � �.
Writing covariance equation in matrix form for
k � � � � � n:�����
r��� r���� � � � r��n�
r��� r��� ...... . . . r����
r�n� � � � r���
�����
�����
�a�
...
an�
������
������
�
...
�
�����
R���
��
���
�
�
These are the Yule–Walker (YW) Equations.
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–5by P. Stoica and R. Moses, Prentice Hall, 1997
AR Spectral Estimation: YW Method
Yule-Walker Method:
Replace r�k� by r�k� and solve for faig and ��:�����
r��� r���� � � � r��n�
r��� r��� ...... . . . r����
r�n� � � � r���
�����
�����
�a�
...
an�
������
������
�
...
�
�����
Then the PSD estimate is
���� �
��
j A���j�
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–6by P. Stoica and R. Moses, Prentice Hall, 1997
AR Spectral Estimation: LS Method
Least Squares Method:
e�t� � y�t� �
nXi��
aiy�t� i� � y�t� � �T �t��
�� y�t� � y�t�
where ��t� � �y�t� ��� � � � � y�t� n�T .
Find � � �a� � � � anT to minimize
f��� �
NXt�n��
je�t�j�
This gives � � ��Y �Y ����Y �y� where
y �
���
y�n���
y�n���
...
y�N�
��� � Y �
���
y�n� y�n� �� � � � y���
y�n��� y�n� � � � y���
......
y�N � �� y�N � �� � � � y�N � n��
��
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–7by P. Stoica and R. Moses, Prentice Hall, 1997
Levinson–Durbin Algorithm
Fast, order-recursive solution to YW equations�����
�� ��� � � � ��n
�� ��
. . . ...... . . . . . . ���
�n � � � �� ��
�����
� �z �
Rn��
�����
��n
������
�����
��n�
...
�
�����
�k � either r�k� or r�k�.
Direct Solution:
� For one given value of n: O�n� flops� For k � �� � � � � n: O�n�� flops
Levinson–Durbin Algorithm:Exploits the Toeplitz form of Rn�� to obtain the solutions
for k � �� � � � � n in O�n�� flops!
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–8by P. Stoica and R. Moses, Prentice Hall, 1997
Levinson-Durbin Alg, con't
Relevant Properties ofR:
� Rx � y � R�x � �y, where �x � �x�n � � � x��
T
� Nested structure
Rn�� �
�� Rn��
��n��
�rn
�n�� �r�n ��
�� � �rn �
��� ��n...
����
��
Thus,
Rn���
� ��n�
���
�� Rn��
��n��
�rn
�n�� �r�n ��
��
�� ��n�
���
�� �n�n
��
where �n � �n����r�n�n
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–9by P. Stoica and R. Moses, Prentice Hall, 1997
Levinson-Durbin Alg, con't
Rn���
� ��n�
�� �
�� �n�n
�� � Rn��
�� ���n
�
���
�� �n��n
��
Combining these gives:
Rn����
��
� ��n�
��� kn
�� ���n
�
��
���
�� �n � kn�n
�
n� kn�n�
��
�� �
n��
��
��
Thus, kn � ��n���n
�n�� �
��n
�
�� kn
���n
�
�
��n�� � ��n � kn��n � ��n��� jknj��
Computation count:
� �k flops for the step k � k��
� n� flops to determine f��k � �kgnk��
This is O�n�� times faster than the direct solution.
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–10by P. Stoica and R. Moses, Prentice Hall, 1997
MA Signals
MA: n� �
y�t� � B�q�e�t�
� e�t� � b�e�t� �� � � � �� bme�t�m�
Thus,
r�k� � � for jkj � m
and
���� � jB���j��� �
mXk��m
r�k�e�i�k
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–11by P. Stoica and R. Moses, Prentice Hall, 1997
MA Spectrum Estimation
Two main ways to Estimate ����:
1. Estimate fbkg and �� and insert them in
���� � jB���j���
� nonlinear estimation problem
� ���� is guaranteed to be� �
2. Insert sample covariances fr�k�g in:
���� �
mXk��m
r�k�e�i�k
� This is �BT ��� with a rectangular lag window of
length �m��.
� ���� is not guaranteed to be� �
Both methods are special cases of ARMA methods
described below, with AR model order n � �.
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–12by P. Stoica and R. Moses, Prentice Hall, 1997
ARMA Signals
ARMA models can represent spectra with both peaks
(AR part) and valleys (MA part).
A�q�y�t� � B�q�e�t�
���� � �������B���A���������
�
Pmk��m ke
�i�k
jA���j�
where
k � E f�B�q�e�t��B�q�e�t � k��g
� E f�A�q�y�t��A�q�y�t � k��g
�
nXj��
nXp��
aja�p r�k� p� j�
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–13by P. Stoica and R. Moses, Prentice Hall, 1997
ARMA Spectrum Estimation
Two Methods:
1. Estimate fai� bj� ��g in ���� � �����B���
A�������
� nonlinear estimation problem; can use an approximate
linear two-stage least squares method
� ���� is guaranteed to be� �
2. Estimate fai� r�k�g in ���� �Pm
k��m �ke�i�k
jA���j�
� linear estimation problem (the Modified Yule-Walker
method).
� ���� is not guaranteed to be� �
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–14by P. Stoica and R. Moses, Prentice Hall, 1997
Two-Stage Least-Squares Method
Assumption: The ARMA model is invertible:
e�t� �
A�q�
B�q�y�t�
� y�t� � ��y�t� �� � ��y�t� �� � � � �
� AR��� with j�kj � � as k ��
Step 1: Approximate, for some large K
e�t� � y�t� � ��y�t� �� � � � �� �Ky�t�K�
1a) Estimate the coefficients f�kgKk�� by using AR
modelling techniques.
1b) Estimate the noise sequence
e�t� � y�t� � ��y�t� �� � � � �� �Ky�t�K�
and its variance
�� �
�
N �K
NXt�K��
je�t�j�Lecture notes to accompany Introduction to Spectral Analysis Slide L4–15by P. Stoica and R. Moses, Prentice Hall, 1997
Two-Stage Least-Squares Method, con't
Step 2: Replace fe�t�g by e�t� in the ARMA equation,
A�q�y�t� � B�q�e�t�
and obtain estimates of fai� bjg by applying least squares
techniques.
Note that the ai and bj coefficients enter linearly in the
above equation:y�t�� e�t� � ��y�t� �� � � �� y�t� n��
e�t� �� � � �e�t�m��
� � �a� � � � an b� � � � bmT
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–16by P. Stoica and R. Moses, Prentice Hall, 1997
Modified Yule-Walker Method
ARMA Covariance Equation:
r�k� �
nXi��
air�k � i� � �� k � m
In matrix form for k � m��� � � � �m�M���
r�m� � � � r�m� n���
r�m��� r�m� n���
... . . . ...
r�m�M � �� � � � r�m� n�M��
���a�
...
an�
� ��
��r�m���
r�m���
...
r�m�M��
��
Replace fr�k�g by fr�k�g and solve for faig.
If M � n, fast Levinson-type algorithms exist for
obtaining faig.
If M � n overdetermined YW system of equations; least
squares solution for faig.
Note: For narrowband ARMA signals, the accuracy of
faig is often better for M � n
Lecture notes to accompany Introduction to Spectral Analysis Slide L4–17by P. Stoica and R. Moses, Prentice Hall, 1997
Su
mm
ary
of
Par
amet
ric
Met
ho
ds
for
Rat
ion
alS
pec
tra
Com
puta
tiona
lG
uara
ntee
Met
hod
Bur
den
Acc
urac
y
� ���
��
�
?U
sefo
rA
R:Y
Wor
LS
low
med
ium
Yes
Spec
tra
with
(nar
row
)pe
aks
but
nova
lley
MA
:BT
low
low
-med
ium
No
Bro
adba
ndsp
ectr
apo
ssib
lyw
ithva
lleys
butn
ope
aks
AR
MA
:MY
Wlo
w-m
ediu
mm
ediu
mN
oSp
ectr
aw
ithbo
thpe
aks
and
(not
too
deep
)va
lleys
AR
MA
:2-S
tage
LS
med
ium
-hig
hm
ediu
m-h
igh
Yes
As
abov
e
Lec
ture
note
sto
acco
mpa
nyIn
trod
uctio
nto
Spec
tral
Ana
lysi
sSl
ide
L4–
18by
P.St
oica
and
R.M
oses
,Pre
ntic
eH
all,
1997
Parametric Methods
for
Line Spectra — Part 1
Lecture 5
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–1by P. Stoica and R. Moses, Prentice Hall, 1997
Line Spectra
Many applications have signals with (near) sinusoidalcomponents. Examples:
� communications
� radar, sonar
� geophysical seismology
ARMA model is a poor approximation
Better approximation by Discrete/Line Spectrum Models
�
�
�
�
�
��
�� ��� �� �
�������
�������������
�
����
An “Ideal” line spectrum
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–2by P. Stoica and R. Moses, Prentice Hall, 1997
Line Spectral Signal Model
Signal Model: Sinusoidal components of frequencies
f�kg and powers f��kg, superimposed in white noise of
power ��.
y�t� � x�t� � e�t� t � ���� � � �
x�t� �
nXk��
�kei��kt��k�� �z �
xk�t�
Assumptions:
A1: �k � � �k � ���� �
(prevents model ambiguities)
A2: f�kg� independent rv's, uniformly
distributed on ���� �
(realistic and mathematically convenient)
A3: e�t� � circular white noise with variance ��
E fe�t�e��s�g � ��t�s E fe�t�e�s�g � �
(can be achieved by “slow” sampling)
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–3by P. Stoica and R. Moses, Prentice Hall, 1997
Covariance Function and PSD
Note that:
� Enei pe�i j
o� �, for p � j
� Enei pe�i j
o� E
nei p
oE
ne�i j
o
�
���� ���Z �
��ei d�
����� � �, for p � j
Hence,E
nxp�t�x�j�t� k�
o� ��p ei�pk p�j
r�k� � E fy�t�y��t� k�g
�
Pnp���
�pe
i�pk � ��k��
and
���� � ��
nXp��
��p�� � �p� � ��
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–4by P. Stoica and R. Moses, Prentice Hall, 1997
Parameter Estimation
Estimate either:
� f�k� �k� �kgnk��� �
� (Signal Model)
� f�k� ��kg
nk��� �
� (PSD Model)
Major Estimation Problem: f��kg
Once f�kg are determined:
� f��kg can be obtained by a least squares method from
r�k� �
nXp��
��pei�pk + residuals
OR:
� Both f�kg and f�kg can be derived by a leastsquares method from
y�t� �
nXk��
�kei�kt + residuals
with �k � �kei k .
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–5by P. Stoica and R. Moses, Prentice Hall, 1997
Nonlinear Least Squares (NLS) Method
min
f�k�k� kg
NXt��
������y�t��nX
k���kei��kt� k�
�������
� �z �
F���� �
Let:�k � �kei k
� � ��� � � � �nT
Y � �y��� � � � y�N�T
B �
��� ei�� � � � ei�n
... ...
eiN�� � � � eiN�n�
��
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–6by P. Stoica and R. Moses, Prentice Hall, 1997
Nonlinear Least Squares (NLS) Method, con't
Then:
F � �Y �B����Y �B�� � kY �B�k�
� �� � �B�B���B�Y ��B�B
�� � �B�B���B�Y
�Y �Y � Y �B�B�B���B�Y
This gives:
� � �B�B���B�Y���
���
and
� � arg max�
Y �B�B�B���B�YLecture notes to accompany Introduction to Spectral Analysis Slide L5–7by P. Stoica and R. Moses, Prentice Hall, 1997
NLS Properties
Excellent Accuracy:
var ��k� �
���
N��k
�for N � ��
Example: N � ���
SNRk � ��k��� � �� dB
Thenq
var��k� � ����.
Difficult Implementation:
The NLS cost function F is multimodal; it is difficult to
avoid convergence to local minima.
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–8by P. Stoica and R. Moses, Prentice Hall, 1997
Unwindowed Periodogram as anApproximate NLS Method
For a single (complex) sinusoid, the maximum of the
unwindowed periodogram is the NLS frequency estimate:
Assume: n � �
Then: B�B � N
B�Y �
NXt��
y�t�e�i�t � Y ��� (finite DTFT)
Y �B�B�B���B�Y �
�N
jY ���j�
� �p���
� (Unwindowed Periodogram)
So, with no approximation,
� � argmax�
�p���
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–9by P. Stoica and R. Moses, Prentice Hall, 1997
Unwindowed Periodogram as anApproximate NLS Method, con't
Assume: n � �
Then:
f�kgnk�� � the locations of the n largest
peaks of �p���
provided that
inf j�k � �pj � ���N
which is the periodogram resolution limit.
If better resolution desired then use a High/SuperResolution method.
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–10by P. Stoica and R. Moses, Prentice Hall, 1997
High-Order Yule-Walker Method
Recall:
y�t� � x�t� � e�t� �
nXk��
�kei��kt� k�� �z �
xk�t�
�e�t�
“Degenerate” ARMA equation for y�t�:
�� � ei�kq���xk�t�
� �knei��kt� k� � ei�k ei �k�t���� k�
o� �
Let
B�q� � ��
LXk��
bkq�k �� A�q� �A�q�
A�q� � ��� ei��q��� � � � ��� ei�nq���
�A�q� � arbitrary
Then B�q�x�t� � �
B�q�y�t� � B�q�e�t�
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–11by P. Stoica and R. Moses, Prentice Hall, 1997
High-Order Yule-Walker Method, con't
Estimation Procedure:
� Estimate fbigLi�� using an ARMA MYW technique
� Roots of B�q� give f�kgnk��, along with L� n
“spurious” roots.
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–12by P. Stoica and R. Moses, Prentice Hall, 1997
High-Order and Overdetermined YW Equations
ARMA covariance:
r�k� �
LXi��
bir�k � i� � �� k � L
In matrix form for k � L��� � � � � L�M
�����
r�L� � � � r���
r�L��� � � � r���
... ...
r�L�M � �� � � � r�M��
����
� �z �
���
b � ��
����r�L���
r�L���
...
r�L�M��
����
� �z ����
This is a high-order (if L � n) and overdetermined
(if M � L) system of YW equations.
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–13by P. Stoica and R. Moses, Prentice Hall, 1997
High-Order and Overdetermined YW Equations,con't
Fact: rank��� � n
SVD of �: � � U�V �
� U � �M � n� with U�U � In
� V � � �n� L� with V �V � In
� � � �n� n�, diagonal and nonsingular
Thus,
�U�V ��b � ��
The Minimum-Norm solution is
b � ��y�� �V���U��
Important property: The additional �L� n� spurious
zeros of B�q� are located strictly inside the unit circle, if
the Minimum-Norm solution b is used.
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–14by P. Stoica and R. Moses, Prentice Hall, 1997
HOYW Equations, Practical Solution
Let � � � but made from fr�k�g instead of fr�k�g.
Let U , �, V be defined similarly to U , �, V from the
SVD of �.
Compute b � �V ���U��
Then f�kgnk�� are found from the n zeroes of B�q� that
are closest to the unit circle.
When the SNR is low, this approach may give spurious
frequency estimates when L � n; this is the price paid for
increased accuracy when L � n.
Lecture notes to accompany Introduction to Spectral Analysis Slide L5–15by P. Stoica and R. Moses, Prentice Hall, 1997
Parametric Methods
for
Line Spectra — Part 2
Lecture 6
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–1by P. Stoica and R. Moses, Prentice Hall, 1997
The Covariance Matrix Equation
Let:
a��� � �� e�i� � � � e�i�m����T
A � �a���� � � � a��n� �m� n�
Note: rank�A� � n (for m � n )
Define
�y�t���
�����
y�t�
y�t� ��
...
y�t�m����
���� � A�x�t� � �e�t�
where
�x�t� � �x��t� � � � xn�t�T
�e�t� � �e�t� � � � e�t�m���T
Then
R�� E f�y�t��y��t�g � APA�� ��I
with
P � E f�x�t��x��t�g �
��� ��� �
. . .
� ��n�
��Lecture notes to accompany Introduction to Spectral Analysis Slide L6–2by P. Stoica and R. Moses, Prentice Hall, 1997
Eigendecomposition of R and Its Properties
R � APA�� ��I �m � n�
Let:
�� � �� � � � � � �m: eigenvalues of R
fs�� � � � sng: orthonormal eigenvectors associated
with f��� � � � � �ng
fg�� � � � � gm�ng: orthonormal eigenvectors associated
with f�n��� � � � � �mg
S � �s� � � � sn �m� n�
G � �g� � � � gm�n �m� �m� n��
Thus,
R � �S G�
�� ��
. . .
�m�
���S�G�
�
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–3by P. Stoica and R. Moses, Prentice Hall, 1997
Eigendecomposition of R and Its Properties,con't
As rank�APA�� � n:
�k � �� k � �� � � � � n
�k � �� k � n��� � � � �m
���
��� �� � �� �
. . .
� �n � ���
�� � nonsingular
Note:
RS � APA�S� ��S � S�
�� �� �
. . .
� �n�
��
S � A�PA�S�� ����� AC
with jCj � � (since rank�S� � rank�A� � n).
Therefore, since S�G� �,
A�G � �
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–4by P. Stoica and R. Moses, Prentice Hall, 1997
MUSIC Method
A�G�
��� a�����
...a���n�
��� G � �
fa��k�gnk�� � R�G�
Thus,
f�kgnk�� are the unique solutions of
a����GG�a��� � �.
Let:
R �
�N
NXt�m
�y�t��y��t�
S� G � S�G made from the
eigenvectors of R
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–5by P. Stoica and R. Moses, Prentice Hall, 1997
Spectral and Root MUSIC Methods
Spectral MUSIC Method:
f�kgnk�� � the locations of the n highest peaks of the
“pseudo-spectrum” function:
�
a���� G G�a���� � � ���� �
Root MUSIC Method:
f�kgnk�� � the angular positions of the n roots of:
aT �z��� G G�a�z� � �
that are closest to the unit circle. Here,
a�z� � ��� z��� � � � � z��m���T
Note: Both variants of MUSIC may produce spuriousfrequency estimates.
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–6by P. Stoica and R. Moses, Prentice Hall, 1997
Pisarenko Method
Pisarenko is a special case of MUSIC with m � n��
(the minimum possible value).
If: m � n��Then: G � g�,
f�kgnk�� can be found from the roots of
aT �z���g� � �
� no problem with spurious frequency estimates
� computationally simple
� (much) less accurate than MUSIC with m� n��
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–7by P. Stoica and R. Moses, Prentice Hall, 1997
Min-Norm Method
Goals: Reduce computational burden, and reduce risk of
false frequency estimates.
Uses m� n (as in MUSIC), but only one vector in
R�G� (as in Pisarenko).
Let ��
g�
� the vector in R� G�, with first element equalto one, that has minimum Euclidean norm.
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–8by P. Stoica and R. Moses, Prentice Hall, 1997
Min-Norm Method, con't
Spectral Min-Norm
f�gnk�� � the locations of the n highest peaks in the
“pseudo-spectrum”� �
�����a������
g�������
Root Min-Normf�gnk�� � the angular positions of the n roots of the
polynomial
aT �z�����
g�
that are closest to the unit circle.
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–9by P. Stoica and R. Moses, Prentice Hall, 1997
Min-Norm Method: Determining �g
Let S �
����S
�g �g m� �
Then: ��
g�� R� G� S�
��
g�
� �
�S�g � ��
Min-Norm solution: g � ��S��S��S����
As: I � S�S � ���� �S��S, ��S��S��� exists iff
��� � k�k� � �
(This holds, at least, for N � �.)
Multiplying the above equation by � gives:
���� k�k�� � ��S��S��
��S��S���� � ����� k�k��
g � ��S����� k�k��Lecture notes to accompany Introduction to Spectral Analysis Slide L6–10by P. Stoica and R. Moses, Prentice Hall, 1997
ESPRIT Method
Let A� � �Im�� �A
A� � �� Im��A
Then A� � A�D, where
D �
��� e�i�� �
. . .
� e�i�n�
��
Also, let S� � �Im�� �S
S� � �� Im��S
Recall S � AC with jCj � �. Then
S� � A�C � A�DC � S�C��DC� �z �
�
So � has the same eigenvalues as D. � is uniquely
determined as
� � �S��S����S��S�
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–11by P. Stoica and R. Moses, Prentice Hall, 1997
ESPRIT Implementation
From the eigendecomposition of R, find S, then S� and
S�.
The frequency estimates are found by:
f�kgnk�� � �arg��k�
where f�kgnk�� are the eigenvalues of
� � �S��S����S��S�
ESPRIT Advantages:
� computationally simple
� no extraneous frequency estimates (unlike in MUSIC
or Min–Norm)
� accurate frequency estimates
Lecture notes to accompany Introduction to Spectral Analysis Slide L6–12by P. Stoica and R. Moses, Prentice Hall, 1997
Su
mm
ary
of
Fre
qu
ency
Est
imat
ion
Met
ho
ds
Com
puta
tion
alA
ccur
acy
/R
isk
for
Fal
seM
etho
dB
urde
nR
esol
utio
nF
req
Est
imat
esPe
riod
ogra
msm
all
med
ium
-hig
hm
ediu
mN
onlin
ear
LS
very
high
very
high
very
high
Yul
e-W
alke
rm
ediu
mhi
ghm
ediu
mPi
sare
nko
smal
llo
wno
neM
USI
Chi
ghhi
ghm
ediu
mM
in-N
orm
med
ium
high
smal
lE
SPR
ITm
ediu
mve
ryhi
ghno
ne
Rec
omm
enda
tion:
�
Use
Per
iodo
gram
for
med
ium
-res
olut
ion
appl
icat
ions
�
Use
ESP
RIT
for
high
-res
olut
ion
appl
icat
ions
Lec
ture
note
sto
acco
mpa
nyIn
trod
uctio
nto
Spec
tral
Ana
lysi
sSl
ide
L6–
13by
P.St
oica
and
R.M
oses
,Pre
ntic
eH
all,
1997
Filter Bank Methods
Lecture 7
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–1by P. Stoica and R. Moses, Prentice Hall, 1997
Basic Ideas
Two main PSD estimation approaches:
1. Parametric Approach: Parameterize ���� by a
finite-dimensional model.
2. Nonparametric Approach: Implicitly smooth
f����g����� by assuming that ���� is nearly
constant over the bands
�� � ��� �� ��� � � �
2 is more general than 1, but 2 requires
N� � �
to ensure that the number of estimated values
(� ������ � ���) is � N .
N� � � leads to the variability / resolution compromise
associated with all nonparametric methods.
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–2by P. Stoica and R. Moses, Prentice Hall, 1997
Filt
erB
ank
Ap
pro
ach
toS
pec
tral
Est
imat
ion
��
��
Div
isio
nb
y�
lter
ban
dw
idth
Pow
erC
alcu
lati
on
Ban
dp
ass
Fil
ter
wit
hva
ryin
g�
can
d�
xed
ban
dw
idth
� ���
c�
Pow
erin
the
ban
dF
ilte
red
Sig
nal
y�t
�
�
�
��
�c
��
�y��
�
��
����
�
�
��
�c
��
��y��
�
� �
�
�
�
�c
�
H��
�
�F
B��
�
(a�
� ��
Z � ��
jH��
�j��
���d
���
(b�
� ��
Z ���
�
��
��
���
�d���
(c�
���
�
(a)
cons
iste
ntpo
wer
calc
ulat
ion
(b)
Idea
lpas
sban
dfil
ter
with
band
wid
th
�
(c)
����
cons
tant
on
�
��
��
���
��
��
�
Not
eth
atas
sum
ptio
ns(a
)an
d(b
),as
wel
las
(b)
and
(c),
are
confl
ictin
g.
Lec
ture
note
sto
acco
mpa
nyIn
trod
uctio
nto
Spec
tral
Ana
lysi
sSl
ide
L7–
3by
P.St
oica
and
R.M
oses
,Pre
ntic
eH
all,
1997
Filter Bank Interpretation of the Periodogram
�p����
��
�N
������NX
t��y�t�e�i��t
�������
�
�N
������NX
t��y�t�ei���N�t�
�������
� N�������X
k��hky�N � k�
�������
wherehk �
��N e
i��k� k � �� � � � � N � �
�� otherwise
H��� �
�Xk��
hke�i�k �
�N
eiN������ � �
ei������ � �
� center frequency of H��� � ��
� 3dB bandwidth of H��� � ��N
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–4by P. Stoica and R. Moses, Prentice Hall, 1997
Filter Bank Interpretation of the Periodogram,con't
jH���j as a function of ��� � ��, for N � ��.
−3 −2 −1 0 1 2 3−40
−35
−30
−25
−20
−15
−10
−5
0
dB
ANGULAR FREQUENCY
Conclusion: The periodogram �p��� is a filter bank PSD
estimator with bandpass filter as given above, and:
� narrow filter passband,
� power calculation from only 1 sample of filter output.
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–5by P. Stoica and R. Moses, Prentice Hall, 1997
Possible Improvements to the Filter BankApproach
1. Split the available sample, and bandpass filter each
subsample.
� more data points for the power calculation stage.
This approach leads to Bartlett and Welch methods.
2. Use several bandpass filters on the whole sample.
Each filter covers a small band centered on ��.
� provides several samples for power calculation.
This “multiwindow approach” is similar to the
Daniell method.
Both approaches compromise bias for variance, and in fact
are quite related to each other: splitting the data sample
can be interpreted as a special form of windowing or
filtering.
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–6by P. Stoica and R. Moses, Prentice Hall, 1997
Capon Method
Idea: Data-dependent bandpass filter design.
yF�t� �
mXk��
hky�t� k�
� �h� h� � � � hm� �z �
h�
��� y�t�
...
y�t�m��
��
� �z �
�y�t�
EnjyF�t�j�o� h�Rh� R � E f�y�t��y��t�g
H��� �
mXk��
hke�i�k � h�a���
where a��� � ��� e�i� � � � e�im�T
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–7by P. Stoica and R. Moses, Prentice Hall, 1997
Capon Method, con't
Capon Filter Design Problem:
minh
�h�Rh� subject to h�a��� � �
Solution: h� � R��a�a�R��a
The power at the filter output is:
EnjyF�t�j�o� h��Rh� � ��a����R��a���
which should be the power of y�t� in a passband centeredon �.
The Bandwidth� �m�� �
�
(filter length)
Conclusion Estimate PSD as:
���� �
m��
a���� R��a���
with
R �
�
N �m
NXt�m��
�y�t��y��t�
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–8by P. Stoica and R. Moses, Prentice Hall, 1997
Capon Properties
� m is the user parameter that controls the compromise
between bias and variance:
– as m increases, bias decreases and variance
increases.
� Capon uses one bandpass filter only, but it splits the
N -data point sample into (N �m) subsequences of
length m with maximum overlap.
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–9by P. Stoica and R. Moses, Prentice Hall, 1997
Relation between Capon and Blackman-TukeyMethods
Consider �BT ��� with Bartlett window:
�BT ��� �
mXk��m
m��� jkj
m��
r�k�e�i�k
�
�m��
mXt��
mXs��
r�t� s�e�i��t�s�
�
a���� Ra���
m��
� R � �r�i� j�
Then we have
�BT ��� �
a���� Ra���
m��
�C��� �
m��
a���� R��a���
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–10by P. Stoica and R. Moses, Prentice Hall, 1997
Relation between Capon and AR Methods
Let
�AR
k ��� �
��k
j Ak���j�
be the kth order AR PSD estimate of y�t�.
Then
�C��� �
�
�m��
mXk��
���AR
k ���
Consequences:
� Due to the average over k, �C��� generally has lessstatistical variability than the AR PSD estimator.
� Due to the low-order AR terms in the average,
�C��� generally has worse resolution and biasproperties than the AR method.
Lecture notes to accompany Introduction to Spectral Analysis Slide L7–11by P. Stoica and R. Moses, Prentice Hall, 1997
Spatial Methods — Part 1
Lecture 8
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–1by P. Stoica and R. Moses, Prentice Hall, 1997
The Spatial Spectral Estimation Problem
Source n
Source �
Source �
�����
BBBBBN
����R
v
v
v
Sensor �
�� ��
�� ��
�� ��
Sensor �
Sensor m
ccccc
cccc
ccc
�����������������
���
����
Problem: Detect and locate n radiating sources by using
an array of m passive sensors.
Emitted energy: Acoustic, electromagnetic, mechanical
Receiving sensors: Hydrophones, antennas, seismometers
Applications: Radar, sonar, communications, seismology,
underwater surveillance
Basic Approach: Determine energy distribution over
space (thus the name “spatial spectral analysis”)
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–2by P. Stoica and R. Moses, Prentice Hall, 1997
Simplifying Assumptions
� Far-field sources in the same plane as the array of
sensors
� Non-dispersive wave propagation
Hence: The waves are planar and the only location
parameter is direction of arrival (DOA)(or angle of arrival, AOA).
� The number of sources n is known. (We do not treat
the detection problem)
� The sensors are linear dynamic elements with knowntransfer characteristics and known locations
(That is, the array is calibrated.)
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–3by P. Stoica and R. Moses, Prentice Hall, 1997
Array Model — Single Emitter Case
x�t� � the signal waveform as measured at a referencepoint (e.g., at the “first” sensor)
�k � the delay between the reference point and the
kth sensor
hk�t� � the impulse response (weighting function) ofsensor k
�ek�t� � “noise” at the kth sensor (e.g., thermal noise insensor electronics; background noise, etc.)
Note: t � R (continuous-time signals).
Then the output of sensor k is
�yk�t� � hk�t� � x�t� �k� � �ek�t�
(� � convolution operator).
Basic Problem: Estimate the time delays f�kg with hk�t�known but x�t� unknown.
This is a time-delay estimation problem in the unknowninput case.
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–4by P. Stoica and R. Moses, Prentice Hall, 1997
Narrowband Assumption
Assume: The emitted signals are narrowband with known
carrier frequency �c.
Then: x�t� � ��t� cos��ct� ��t�
where ��t�� ��t� vary “slowly enough” so that
��t� �k� � ��t�� ��t� �k� � ��t�
Time delay is now� to a phase shift �c�k:
x�t� �k� � ��t� cos��ct� ��t�� �c�k
hk�t� � x�t� �k�
jHk��c�j�t� cos �ct� �t�� �c�k �argfHk��c�g�
where Hk��� � Ffhk�t�g is the kth sensor's transfer
function
Hence, the kth sensor output is
�yk�t� � jHk��c�j��t�
� cos��ct� ��t�� �c�k � argHk��c� � �ek�t�
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–5by P. Stoica and R. Moses, Prentice Hall, 1997
Complex Signal Representation
The noise-free output has the form:
z�t� � ��t� cos ��ct� ��t� �
�
��t��
nei �ct���t��� e�i �ct���t��
o
Demodulate z�t� (translate to baseband):
�z�t�e��ct � ��t�f ei��t�� �z �
lowpass
� e�i ��ct���t��� �z �
highpass
g
Lowpass filter �z�t�e�i�ct to obtain ��t�ei��t�
Hence, by low-pass filtering and sampling the signal
�yk�t��� � �yk�t�e�i�ct
� �yk�t� cos��ct�� i�yk�t� sin��ct�
we get the complex representation: (for t � Z)
yk�t� � �t� ei��t�� �z �s�t�
jHk��c�j eiargHk��c�� �z �
Hk��c�
e�i�c�k � ek�t�or
yk�t� � s�t�Hk��c� e�i�c�k � ek�t�
where s�t� is the complex envelope of x�t�.
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–6by P. Stoica and R. Moses, Prentice Hall, 1997
Vector Representation for a Narrowband Source
Let
� � the emitter DOA
m � the number of sensors
a��� �
��� H���c� e
�i�c��
...
Hm��c� e�i�c�m�
��
y�t� �
��� y��t�
...
ym�t��
�� e�t� �
��� e��t�
...
em�t��
��
Then
y�t� � a���s�t� � e�t�
NOTE: � enters a��� via both f�kg and fHk��c�g.
For omnidirectional sensors the fHk��c�g do not depend
on �.
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–7by P. Stoica and R. Moses, Prentice Hall, 1997
An
alo
gP
roce
ssin
gB
lock
Dia
gra
m
Ana
log
proc
essi
ngfo
rea
chre
ceiv
ing
arra
yel
emen
t
XX
XXXXXz
��
������
Exte
rnal
Noi
se
Inco
min
gSig
nal
x�t
��
k�
SE
NSO
R
���
lll
�C
abli
ng
and
Fil
teri
ng
�
Inte
rnal
Noi
se
Tra
nsd
uce
r
�y
k�t
�
DE
MO
DU
LA
TIO
N
����
������
Osc
illa
tor
Low
pas
sF
ilte
r
Low
pas
sF
ilte
r� �
� ��s
in��
ct�
cos�
�ct�
� �Im
�yk�t
��
Re�
yk�t
��� �
Im��y
k�t
��
Re�
�y k�t
��
A�D
CO
NV
ER
SIO
N
��
��
� �
A�D
A�D
Lec
ture
note
sto
acco
mpa
nyIn
trod
uctio
nto
Spec
tral
Ana
lysi
sSl
ide
L8–
8by
P.St
oica
and
R.M
oses
,Pre
ntic
eH
all,
1997
Multiple Emitter Case
Given n emitters with
� received signals: fsk�t�gnk��
� DOAs: �k
Linear sensors
y�t� � a����s��t� � � � �� a��n�sn�t� � e�t�
Let
A � �a���� � � � a��n�� �m� n�
s�t� � �s��t� � � � sn�t�T � �n� ��
Then, the array equation is:
y�t� � As�t� � e�t�
Use the planar wave assumption to find the dependence of
�k on �.
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–9by P. Stoica and R. Moses, Prentice Hall, 1997
Uniform Linear Arrays
�����������������������
Source
�
�
�
���
p p p
Sensor
m����
d sin �JJ�JJ�
d ��
�����������
JJJJJJJJJJJJJJJJJJJJJ
JJJJJ�
h
v v v v v
ULA Geometry
Sensor #1 = time delay reference
Time Delay for sensor k:
�k � �k � ��d sin �
c
where c = wave propagation speed
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–10by P. Stoica and R. Moses, Prentice Hall, 1997
Spatial Frequency
Let:
�s
�� �c
d sin �c
� ��d sin �
c�fc� ��
d sin ��
� � c�fc � signal wavelength
a��� � ��� e�i�s � � � e�i�m����sT
By direct analogy with the vector a��� made from
uniform samples of a sinusoidal time series,
�s � spatial frequency
The function �s �� a��� is one-to-one for
j�sj � �
dj sin �j
���
�� d ���
As
d � spatial sampling period
d ��� is a spatial Shannon sampling theorem.
Lecture notes to accompany Introduction to Spectral Analysis Slide L8–11by P. Stoica and R. Moses, Prentice Hall, 1997
Spatial Methods — Part 2
Lecture 9
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–1by P. Stoica and R. Moses, Prentice Hall, 1997
Spatial Filtering
Spatial filtering useful for
� DOA discrimination (similar to frequency
discrimination of time-series filtering)
� Nonparametric DOA estimation
There is a strong analogy between temporal filtering and
spatial filtering.
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–2by P. Stoica and R. Moses, Prentice Hall, 1997
Analogy between Temporal and Spatial Filtering
Temporal FIR Filter:
yF�t� �
m��Xk��
hku�t� k� � h�y�t�
h � �ho � � � hm���
y�t� � �u�t� � � � u�t�m���T
If u�t� � ei�t then
yF�t� � �h�a���� �z �
filter transfer function
u�t�
a��� � ��� e�i� � � � e�i�m����T
We can select h to enhance or attenuate signals with
different frequencies �.
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–3by P. Stoica and R. Moses, Prentice Hall, 1997
Analogy between Temporal and Spatial Filtering
Spatial Filter:
fyk�t�gmk�� � the “spatial samples” obtained with a
sensor array.
Spatial FIR Filter output:
yF�t� �
mXk��
hkyk�t� � h�y�t�
Narrowband Wavefront: The array's (noise-free)
response to a narrowband (� sinusoidal) wavefront with
complex envelope s�t� is:
y�t� � a���s�t�
a��� � ��� e�i�c�� � � � e�i�c�mT
The corresponding filter output is
yF�t� � �h�a���� �z �
filter transfer function
s�t�
We can select h to enhance or attenuate signals coming
from different DOAs.
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–4by P. Stoica and R. Moses, Prentice Hall, 1997
Analogy between Temporal and Spatial Filtering
�Temporal sampling�
z
�
qqq
��
��
��
��
��
���
�
��
�
����
����
����
����
�����R
�
��
qqq
�s
�
�s
��
�
P
u�t� � ei�t
q��
q��
�
m� �
�BBBBBBBBBBBBBB�
�e�i�
���
e�i�m�����
CCCCCCCCCCCCCCA
� �z �
a���
u�t�
h�h�
hm��
�h�a����u�t�
(a) Temporal filter
ZZZZZ�
����
�����
���
narrowband source with DOA��
�Spatial sampling�
��
�
P
z
�
qqq
��
��
��
��
��
���
�����R�
��
�
����
����
����
����
���aa
���aa
���aa
qqq
reference point
��
m
�BBBBBBBBBBB�
�e�i������
���
e�i��m���
�CCCCCCCCCCCA
� �z �
a���
s�t�
h�h�
hm��
�h�a����s�t�
(b) Spatial filter
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–5by P. Stoica and R. Moses, Prentice Hall, 1997
Spatial Filtering, con't
Example: The response magnitude jh�a���j of a spatial
filter (or beamformer) for a 10-element ULA. Here,
h � a����, where �� � ���
2 4 6 8 100
Magnitude
Theta (deg)
−90
−60
−30
0
30
60
90
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–6by P. Stoica and R. Moses, Prentice Hall, 1997
Spatial Filtering Uses
Spatial Filters can be used
� To pass the signal of interest only, hence filtering out
interferences located outside the filter's beam (but
possibly having the same temporal characteristics as
the signal).
� To locate an emitter in the field of view, by sweeping
the filter through the DOA range of interest
(“goniometer”).
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–7by P. Stoica and R. Moses, Prentice Hall, 1997
Nonparametric Spatial Methods
A Filter Bank Approach to DOA estimation.
Basic Ideas
� Design a filter h��� such that for each �
– It passes undistorted the signal with DOA = �
– It attenuates all DOAs � �
� Sweep the filter through the DOA range of interest,
and evaluate the powers of the filtered signals:
EnjyF�t�j�o� E
njh����y�t�j�
o
� h����Rh���
with R � E fy�t�y��t�g.
� The (dominant) peaks of h����Rh��� give the
DOAs of the sources.
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–8by P. Stoica and R. Moses, Prentice Hall, 1997
Beamforming Method
Assume the array output is spatially white:
R � E fy�t�y��t�g � I
Then: EnjyF �t�j�
o� h�h
Hence: In direct analogy with the temporally white
assumption for filter bank methods, y�t� can be
considered as impinging on the array from all DOAs.
Filter Design:
minh
�h�h� subject to h�a��� � �
Solution:
h � a����a����a��� � a����m
EnjyF�t�j�o� a����Ra����m�
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–9by P. Stoica and R. Moses, Prentice Hall, 1997
Implementation of Beamforming
R �
�N
NXt��
y�t�y��t�
The beamforming DOA estimates are:
f�kg� the locations of the n largest peaks of
a���� Ra���.
This is the direct spatial analog of the Blackman-Tukey
periodogram.
Resolution Threshold:
inf j�k � �pj �
wavelength
array length
� array beamwidth
Inconsistency problem:Beamforming DOA estimates are consistent if n� �, but
inconsistent if n � �.
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–10by P. Stoica and R. Moses, Prentice Hall, 1997
Capon Method
Filter design:
minh
�h�Rh� subject to h�a��� � �
Solution:
h � R��a����a����R��a���
EnjyF�t�j�o� ��a����R��a���
Implementation:
f�kg � the locations of the n largest peaks of
��a���� R��a����
Performance: Slightly superior to Beamforming.
Both Beamforming and Capon are nonparametricapproaches. They do not make assumptions on the
covariance properties of the data (and hence do not
depend on them).
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–11by P. Stoica and R. Moses, Prentice Hall, 1997
Parametric Methods
Assumptions:
� The array is described by the equation:
y�t� � As�t� � e�t�
� The noise is spatially white and has the same power inall sensors:
E fe�t�e��t�g � ��I
� The signal covariance matrix
P � E fs�t�s��t�g
is nonsingular.
Then:
R � E fy�t�y��t�g � APA�� ��I
Thus: The NLS, YW, MUSIC, MIN-NORM and ESPRITmethods of frequency estimation can be used, almostwithout modification, for DOA estimation.
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–12by P. Stoica and R. Moses, Prentice Hall, 1997
Nonlinear Least Squares Method
min
f�kg� fs�t�g
�N
NXt��
ky�t��As�t�k�
� �z �
f���s�
Minimizing f over s gives
s�t� � �A�A���A�y�t�� t � �� � � � � N
Then
f���s� �
�N
NXt��
k �I �A�A�A���A�y�t�k�
�
�N
NXt��
y��t��I �A�A�A���A�y�t�
� trf�I �A�A�A���A� Rg
Thus, f�kg� argmax
f�kg
trf�A�A�A���A� Rg
For N � �, this is precisely the form of the NLS method
of frequency estimation.
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–13by P. Stoica and R. Moses, Prentice Hall, 1997
Nonlinear Least Squares Method
Properties of NLS:
� Performance: high
� Computational complexity: high� Main drawback: need for multidimensional search.
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–14by P. Stoica and R. Moses, Prentice Hall, 1997
Yule-Walker Method
y�t� �
��y�t�
�y�t��
�
��A
�A�
s�t� ���e�t�
�e�t��
Assume: E f�e�t��e��t�g � �
Then:
��� E f�y�t��y��t�g � �AP �A� �M � L�
Also assume:
� M � n� L � n � m �M � L � �n�
� rank� �A� � rank� �A� � n
Then: rank��� � n, and the SVD of � is
� � �U���z�n
U���z�M�n
��n�n �
� �� �V ��V ��
�g ng L�n
Properties: �A�V� � � V� � R� �A�Lecture notes to accompany Introduction to Spectral Analysis Slide L9–15by P. Stoica and R. Moses, Prentice Hall, 1997
YW-MUSIC DOA Estimator
f�kg� the n largest peaks of
���a����V�V���a���
where
� �a���, �L� ��, is the “array transfer vector” for �y�t�
at DOA �
� V� is defined similarly to V�, using
� �
�N
NXt��
�y�t��y��t�
Properties:
� Computational complexity: medium
� Performance: satisfactory if m� �n
� Main advantages:
– weak assumption on fe�t�g
– the subarray �A need not be calibrated
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–16by P. Stoica and R. Moses, Prentice Hall, 1997
MUSIC and Min-Norm Methods
Both MUSIC and Min-Norm methods for frequency
estimation apply with only minor modifications to the
DOA estimation problem.
� Spectral forms of MUSIC and Min-Norm can be used
for arbitrary arrays
� Root forms can be used only with ULAs
� MUSIC and Min-Norm break down if the source
signals are coherent; that is, if
rank�P� � rank�E fs�t�s��t�g� � n
Modifications that apply in the coherent case exist.
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–17by P. Stoica and R. Moses, Prentice Hall, 1997
ESPRIT Method
Assumption: The array is made from two identicalsubarrays separated by a known displacement vector.
Let
�m � # sensors in each subarray
A� � �I�m �A (transfer matrix of subarray 1)
A� � �� I�mA (transfer matrix of subarray 2)
Then A� � A�D, whereD �
��� e�i�c����� �
. . .
� e�i�c���n��
��
���� � the time delay from subarray 1 to
subarray 2 for a signal with DOA = �:
���� � d sin����c
where d is the subarray separation and � is measured from
the perpendicular to the subarray displacement vector.
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–18by P. Stoica and R. Moses, Prentice Hall, 1997
ESPRIT Method, con't
ESPRIT Scenario
subarray 1
subarray 2
source
θ
known displacement vector
Properties:
� Requires special array geometry
� Computationally efficient
� No risk of spurious DOA estimates
� Does not require array calibration
Note: For a ULA, the two subarrays are often the first
m�� and last m�� array elements, so �m � m�� and
A� � �Im�� �A� A� � �� Im��A
Lecture notes to accompany Introduction to Spectral Analysis Slide L9–19by P. Stoica and R. Moses, Prentice Hall, 1997
Top Related