Lecture 7 Frequency Domain Methods - web.stanford.edu
Transcript of Lecture 7 Frequency Domain Methods - web.stanford.edu
![Page 1: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/1.jpg)
Lecture 7Frequency Domain Methods
Dennis SunStats 253
July 14, 2014
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 2: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/2.jpg)
Outline of Lecture
1 The Frequency Domain
2 (Discrete) Fourier Transform
3 Spectral Analysis
4 Projects
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 3: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/3.jpg)
The Frequency Domain
Where are we?
1 The Frequency Domain
2 (Discrete) Fourier Transform
3 Spectral Analysis
4 Projects
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 4: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/4.jpg)
The Frequency Domain
A Time Series
0.00 0.05 0.10 0.15 0.20
−1.
0−
0.5
0.0
0.5
1.0
Time
Am
plitu
de
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 5: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/5.jpg)
The Frequency Domain
A Time Series
.2 cos(2π · 196t)
+
0.00 0.05 0.10 0.15 0.20
−1.
0−
0.5
0.0
0.5
1.0
Time
Am
plitu
de
= .5 cos(2π · 294t)
+
.3 cos(2π · 470t)
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 6: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/6.jpg)
The Frequency Domain
Recovering the Weights
Suppose we knew that the only frequencies in the sound were 196, 294,and 470 Hz and we wanted to know the weights.y(t1)
...y(tn)
=
cos(2π · 196t1)...
cos(2π · 196tn)
λ1+
cos(2π · 294t1)...
cos(2π · 294t1)
λ2+
cos(2π · 470t1)...
cos(2π · 470t1)
λ3
This is equivalent toy(t1)...
y(tn)
=
cos(2π · 196t1) cos(2π · 294t1) cos(2π · 490t1)...
......
cos(2π · 196tn) cos(2π · 294tn) cos(2π · 490tn)
λ1λ2λ3
Can write this as y = Aλ and solve by least squares.
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 7: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/7.jpg)
The Frequency Domain
Harmonic Regression
This is called harmonic regression.
Call:
lm(formula = y ~ cos.196 + cos.294 + cos.470 - 1)
Coefficients:
cos.196 cos.294 cos.470
0.2 0.5 0.3
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 8: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/8.jpg)
The Frequency Domain
Transforming to the Frequency Domain
y = Aλ
• What if we don’t know the frequencies?
• We can try to include as many sinusoids cos(fkt) in A as possible.
• Since y contains n observations, A can be at most n× n.
• Now A is full rank, so it is invertible and we also have
λ = A−1y
• λ is an equivalent representation of the signal in the frequencydomain. (y is the signal in the time domain.)
• A is a transform that maps λ→ y. A−1 is the inverse transform.
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 9: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/9.jpg)
The Frequency Domain
Why is the frequency domain relevant for sound?
Because the ear is a frequency domain analyzer!
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 10: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/10.jpg)
(Discrete) Fourier Transform
Where are we?
1 The Frequency Domain
2 (Discrete) Fourier Transform
3 Spectral Analysis
4 Projects
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 11: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/11.jpg)
(Discrete) Fourier Transform
Why the Fourier Transform
• In general, calculating λ = A−1y requires O(n2) operations
• For special choices of A, it’s possible to do it in O(n log n) operations.
• For example, we might choose A to contain the complex exponentials
A =
ejf1t1 · · · ejfnt1
......
ejf1tn · · · ejfntn
, j =√−1.
This is called the Discrete Fourier Transform (DFT).
• Note: ejfkti = cos(fkti) + j sin(fkti)
• The fast algorithm for computing the DFT is called the Fast FourierTransform (FFT).
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 12: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/12.jpg)
(Discrete) Fourier Transform
The Fourier Transform
DFT : λ(fk) =1
n
n∑i=1
y(ti)e−jfkti λ = A−1y
Inverse DFT : y(ti) =
n∑k=1
λ(fk)ejfkti y = Aλ
• The frequencies fk and times ti depend on the sampling rate fs.• For example, CDs sample at 44.1 kHz, so t1 = 0, t2 = 1/44100.• ti = i/fs, fk = fs · 2πk/n• The “unitless” form of the DFT might be easier to work with
conceptually, but you have to add the units back in at the end:
DFT : λk =1
n
n∑i=1
yie−j2πki/n
Inverse DFT : yi =
n∑k=1
λkej2πki/n
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 13: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/13.jpg)
(Discrete) Fourier Transform
The Fourier Transform
• Remember: The A matrix contains complex numbers. So thefrequency domain representation λ = A−1y is also complex-valued.
• For interpretability, we often look at the magnitudes. Ifλk = ak + jbk, then
|λk| =√a2k + b2k.
• Note that y = Aλ must be real-valued. This imposes constraints onλ.
• Let’s hack around in R: abs(fft(y))
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 14: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/14.jpg)
(Discrete) Fourier Transform
Application to Seasonality Estimation
Wolfer sunspot data
1700 1750 1800 1850 1900 1950
050
100
150
Year
Num
ber
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 15: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/15.jpg)
(Discrete) Fourier Transform
Application to Seasonality Estimation
Wolfer sunspot data: plot(abs(fft(sunspot)))
0.0 0.1 0.2 0.3 0.4 0.5
010
0020
0030
0040
0050
0060
00
Frequency (per year)
Mag
nitu
de
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 16: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/16.jpg)
(Discrete) Fourier Transform
Application to Seasonality Estimation
Wolfer sunspot data: Plot against period p = 1/f instead of frequency.
0 20 40 60 80 100 120 140
010
0020
0030
0040
0050
0060
00
Period (years)
Mag
nitu
de
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 17: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/17.jpg)
(Discrete) Fourier Transform
Application to Seasonality Estimation
Wolfer sunspot data:p <- 1 / ((which(lambda == max(lambda[2:n]))-1)/n)
0 20 40 60 80 100 120 140
010
0020
0030
0040
0050
0060
00
Period (years)
Mag
nitu
de
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 18: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/18.jpg)
(Discrete) Fourier Transform
Summary
• We now have a new representation of data, which is sometimes moreenlightening than the time domain.
• We obtain this by taking the DFT and looking at the magnitudes ofthe resulting coefficients.
• We use the DFT (as opposed to some other transform) because itcan be computed efficiently using the FFT.
• There is a 2D version of the DFT for spatial data.
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 19: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/19.jpg)
Spectral Analysis
Where are we?
1 The Frequency Domain
2 (Discrete) Fourier Transform
3 Spectral Analysis
4 Projects
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 20: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/20.jpg)
Spectral Analysis
Random Processes
• We’ve been using the Fourier transform to decompose a function (i.e.,the trend term in yt = µt + εt).
• Can we use it to study a random process εt?
• Let’s do some R simulations.
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 21: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/21.jpg)
Spectral Analysis
Power Spectral Density
• One way to obtain a stationary random process is to take a linearcombination of sinusoids, i.e.,
y(t) =
n∑k=1
λ(fk)ejfkt
where λ(fk) are independent N(0, s(fk)).• The autocorrelation function is
C(h) = E[y(t+ h)y(t)] = E
[(n∑
k=1
λ(fk)ejfk(t+h)
)(n∑
`=1
λ(f`)e−jf`t
)]
=
n∑k=1
n∑`=1
E(λ(fk)λ(f`))ej(fk−f`)tejfkh =
n∑k=1
E(λ2(fk))︸ ︷︷ ︸s(fk)
ejfkh
• The autocorrelation function C(h) is a Fourier pair with s(f), whichis called the power spectral density.
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 22: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/22.jpg)
Spectral Analysis
Spectral Representation Theorem
The spectral representation theorem says that all stationary processeshave this representation (at least in continuous time):
y(t) =
∫ejftdΛ(f)
where Λ is a random zero-mean process with independent increments.
The power spectral density s is the Fourier transform of theautocorrelation function.
s(f) =
∫C(h)e−jfh dh
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 23: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/23.jpg)
Spectral Analysis
Spectral Density Estimation
How do we estimate s(f) given samples y(ti), i = 1, ..., n?
• Sample PSD: Calculate autocorrelations and take Fourier transform.
s(f) =1
n
n−1∑h=−n+1
C(h)e−jfh
where C(h) =1
n− |h|∑i
yiyi+h.
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 24: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/24.jpg)
Spectral Analysis
Spectral Density Estimation
How do we estimate s(f) given samples y(ti), i = 1, ..., n?
• Periodogram: Take Fourier transform and calculate magnitudessquared.
p(f) =
∣∣∣∣∣ 1nn∑
i=1
yie−jfti
∣∣∣∣∣2
=
(1
n
n∑i=1
yie−jfti
)(1
n
n∑m=1
yme−jftm
)
=1
n
n∑i=1
1
n
n∑m=1
yiyme−jf(i−m)/fs
=1
n
n−1∑h=−n+1
[1
n
∑m
ym+hym
]︸ ︷︷ ︸
(n−|h|)n C(h)
e−jfh/fs
• Theorem: As n→∞, s(f), p(f)⇒ s(f)χ22/2.
• So neither s or p estimates s(f) consistently.
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 25: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/25.jpg)
Spectral Analysis
Periodogram Smoothing
Very simple solution: smooth the periodogram.
Let Nf = {k : |fk − f | ≤ B} be all DFT frequencies that are within abandwidth B of f . Then:
psmooth(f) =1
|Nf |∑k∈Nf
p(fk)
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 26: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/26.jpg)
Projects
Where are we?
1 The Frequency Domain
2 (Discrete) Fourier Transform
3 Spectral Analysis
4 Projects
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 27: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/27.jpg)
Projects
Project Proposals
• Project proposals are due Friday.
• Remember: Goal is to do something useful.
• Please make clear in your project proposal what you plan to do withthis project (i.e., publish a paper, release an R package, etc.).
• I will send out an (anonymous) survey about the class. When youcomplete that survey, you will see a link to a form to submit theproject proposal.
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 28: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/28.jpg)
Projects
Project Ideas
• Covariance modeling with kriging that exploits sparse matrixstructure.
• Using spectral density estimation to estimate ARMA parameters.
• Next class: music applications
Dennis Sun Stats 253 – Lecture 7 July 14, 2014
![Page 29: Lecture 7 Frequency Domain Methods - web.stanford.edu](https://reader033.fdocuments.in/reader033/viewer/2022042421/625feafd74d66a770a342209/html5/thumbnails/29.jpg)
Projects
Administrivia
• Graded Homework 1’s will be returned now. Solutions posted.
• Please turn in Homework 2.
• Homework 3 will be posted in a few hours. This one is a predictioncompetition using kriging methods!
• Don’t forget about the project proposal.
Dennis Sun Stats 253 – Lecture 7 July 14, 2014