Environmental Data Analysis with MatLab Lecture 20: Coherence; Tapering and Spectral Analysis.
Environmental Data Analysis with MatLab
description
Transcript of Environmental Data Analysis with MatLab
![Page 1: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/1.jpg)
Environmental Data Analysis with MatLab
Lecture 13:
Filter Theory
![Page 2: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/2.jpg)
Lecture 01 Using MatLabLecture 02 Looking At DataLecture 03 Probability and Measurement Error Lecture 04 Multivariate DistributionsLecture 05 Linear ModelsLecture 06 The Principle of Least SquaresLecture 07 Prior InformationLecture 08 Solving Generalized Least Squares ProblemsLecture 09 Fourier SeriesLecture 10 Complex Fourier SeriesLecture 11 Lessons Learned from the Fourier TransformLecture 12 Power Spectral DensityLecture 13 Filter Theory Lecture 14 Applications of Filters Lecture 15 Factor Analysis Lecture 16 Orthogonal functions Lecture 17 Covariance and AutocorrelationLecture 18 Cross-correlationLecture 19 Smoothing, Correlation and SpectraLecture 20 Coherence; Tapering and Spectral Analysis Lecture 21 InterpolationLecture 22 Hypothesis testing Lecture 23 Hypothesis Testing continued; F-TestsLecture 24 Confidence Limits of Spectra, Bootstraps
SYLLABUS
![Page 3: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/3.jpg)
purpose of the lecture
develop the
Linear Filter
as a way to describe the way past events influence present-time observations
![Page 4: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/4.jpg)
sometimes, no past history is needed
Flame with time-varying heat h(t)Thermometer measuring temperature θ(t)
Flame instantaneously heats the thermometer
Thermometer retains no heat θ(t) h(t)
![Page 5: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/5.jpg)
this idea described as a linear model
θ1= g 1 h 1 . . .θ10= g 1 h 10θ11= g 1 h11θ12= g 1 h 12. . .θN= g 1 h N where g 1 is a constant
![Page 6: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/6.jpg)
But sometimes,past history is needed
Flame with time-varying heat h(t)
Thermometer measuring temperature θ(t)
Heats takes time to seep through plate
Plate retains heatθ(t=t’) history of h(t) for time t<t’
Steel plate
![Page 7: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/7.jpg)
But sometimes,past history is needed
Flame with time-varying heat h(t)
Thermometer measuring temperature q(t)
Heats takes time to seep through plate
Plate retains heat
q(t=t’) history of h(t) for time t<t’
Steel plate “causal”: only past effects present (not future)
![Page 8: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/8.jpg)
this idea described as a linear model
. . .θ10= g 1 h 10 + g 2 h 9 + g 3 h 8 + g 4 h 7 + …θ11= g 1 h 11 + g 2 h 10 + g 3 h 9 + g 4 h 8 + …θ12= g 1 h 12 + g 2 h 11 + g 3 h 10 + g 4 h 9 + …. . .where g 1, g 2, g 3, … are constants called a “filter”
![Page 9: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/9.jpg)
Special case: temperature depends only on the time elapsed since the flame was turned on, and not on the experiment as performed on Monday or Wednesday
same shape
q(t)
q(t)
experiment is “time-shift invariant”
![Page 10: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/10.jpg)
this idea described as a linear model
. . .θ10= g 1 h 10 + g 2 h 9 + g 3 h 8 + g 4 h 7 + …θ11= g 1 h 11 + g 2 h 10 + g 3 h 9 + g 4 h 8 + …θ12= g 1 h 12 + g 2 h 11 + g 3 h 10 + g 4 h 9 + …. . .where g 1, g 2, g 3, … are constants called a filter
coefficients all the same …filter is time-shift invariant …
![Page 11: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/11.jpg)
this idea written as a summation
![Page 12: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/12.jpg)
this idea written as a summation
inputoutput
filter
“convolution”, not multiplication
![Page 13: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/13.jpg)
this idea written as matrix equation
output inputfilter
![Page 14: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/14.jpg)
we’ve heard the word “convolution” beforein Lecture 11
it’s the name of this integral
![Page 15: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/15.jpg)
but the integral can be approximated as the summation we’ve just seen
![Page 16: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/16.jpg)
so, mathematically, what we’re doingis
“convolution”
![Page 17: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/17.jpg)
convolutions can be written two ways. . .θ10= g 1 h 10 + g 2 h 9 + g 3 h 8 + g 4 h 7 + …θ11= g 1 h 11 + g 2 h 10 + g 3 h 9 + g 4 h 8 + …θ12= g 1 h 12 + g 2 h 11 + g 3 h 10 + g 4 h 9 + …. . . g inside matrix
h inside matrix
![Page 18: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/18.jpg)
implying that the convolution operation is symmetric
g*h = h*g
![Page 19: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/19.jpg)
meaning of the filter g
suppose the input is a spikeh = [1, 0, 0, 0 … 0]T
then the output is the filteroutput
input filter
![Page 20: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/20.jpg)
so the filter represents the
“impulse response”
of the experiment
![Page 21: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/21.jpg)
time , t, after impulse
q(t),
K
0
time , t, after impulse
h(t),
W
0
A) Input is spike
B) Output is impulse response
spike
![Page 22: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/22.jpg)
h(t0)g(t-t0)
h(t),
W
time, t
t0
q(t).
K
time, tt0
spike of amplitude, h(t0)
A)
B)
![Page 23: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/23.jpg)
example: heat-generating layer
soil
soil heat-generating layer
observation point
z=0
z-axis
z
![Page 24: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/24.jpg)
known impulse response
based on known physics of heat transfer
![Page 25: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/25.jpg)
known impulse response
soil density1500 kg/m3 soil heat capacity,800 J/kg-K soil thermal diffusivity.1.25×10-6 m2/s
distance from center of layer
![Page 26: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/26.jpg)
0 50 100 150 200 250 300 350 400 450 5000
5
x 10-3
time, days
g(t)
0 50 100 150 200 250 300 350 400 450 500-10
0
10
time, days
htru
e(t)
0 50 100 150 200 250 300 350 400 450 5000
0.5
time, days
qtru
e1(t)
Dt g
(t)htr
ue(t)
, W/m
2q
true
(t), K
time t, days
time t, days
A)
B)
C) time t, days
![Page 27: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/27.jpg)
The Method
solve
using least-squares… well, use damped least square, just in case
![Page 28: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/28.jpg)
0 50 100 150 200 250 300 350 400 450 5000
0.5
time, days
qobs
(t)
0 50 100 150 200 250 300 350 400 450 500-10
0
10
time, days
htru
e(t)
0 50 100 150 200 250 300 350 400 450 500-10
0
10
time, days
hest
(t)q
obs (t
), K
htrue
(t), W
/m2 B)
C)
hest (t
) ),
W/m
2
time t, days
time t, days
time t, days
A)
![Page 29: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/29.jpg)
0 50 100 150 200 250 300 350 400 450 5000
0.5
time, days
qobs
(t)
0 50 100 150 200 250 300 350 400 450 500-10
0
10
time, days
htru
e(t)
0 50 100 150 200 250 300 350 400 450 500-10
0
10
time, days
hest
(t)q
obs (t
), K
htrue
(t), W
/m2 B)
C)
hest (t
) ),
W/m
2
time t, days
time t, days
time t, days
A)
noise in data …
… is amplified
![Page 30: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/30.jpg)
Try adding
prior information of smoothness
(minimize second derivative)
![Page 31: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/31.jpg)
0 50 100 150 200 250 300 350 400 450 5000
0.5
time, days
qobs
(t)
0 50 100 150 200 250 300 350 400 450 500-10
0
10
time, days
htru
e(t)
0 50 100 150 200 250 300 350 400 450 500-10
0
10
time, days
hest
(t)q
obs (t
), K
htrue
(t), W
/m2 B)
C)
hest (t
) ), W
/m2
time t, days
time t, days
time t, days
A)
much less noise
![Page 32: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/32.jpg)
Tips onfilter calculations with MatLab
![Page 33: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/33.jpg)
you should avoid constructing the matrix
because it contains so many redundant elements
G =
![Page 34: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/34.jpg)
Tip #1
use the conv() function to calculate the convolution q = g * h
not the matrix multiplication q=Ghtmp = conv(g, h); q=tmp(1:N);
note that we truncate the output of conv(), so that is has the same length, N, as the input time series
![Page 35: Environmental Data Analysis with MatLab](https://reader035.fdocuments.in/reader035/viewer/2022070419/56815c2c550346895dca059c/html5/thumbnails/35.jpg)
Tip #2
use the bicg() solver, together with the function filterrfun() to solve Fq=h by generalized least
squares not matrix division hest=(FTF)\(FTq)
it implements the matrix multiplications, GT(Gv), using the timeseries g, and does not ever construct G
filterrfun() is described in the text