Negative Power Law Noise, Reality vs Myth
description
Transcript of Negative Power Law Noise, Reality vs Myth
Negative Power Law Noise,Negative Power Law Noise, Reality vs Myth Reality vs Myth
Victor S. ReinhardtVictor S. ReinhardtRaytheon Space and Airborne SystemsRaytheon Space and Airborne Systems
El Segundo, CA, USAEl Segundo, CA, USA
Precise Time and Time IntervalPrecise Time and Time IntervalSystems and Applications MeetingSystems and Applications Meeting
20092009
PTTI 2009 -- V. S. Reinhardt Page 2
Negative Power Law (Neg-p) Noise,Reality vs Myth• Not questioning the Not questioning the realityreality that that -variances-variances
Like Allan and Hadamard variancesLike Allan and Hadamard variancesAre convergent measures of neg-p noiseAre convergent measures of neg-p noise
• But will show it is But will show it is mythmyth that neg-p divergences that neg-p divergences in other variances like standard & N-sample in other variances like standard & N-sample Are mathematical defects in these variances Are mathematical defects in these variances That should be “fixed” by replacing them with That should be “fixed” by replacing them with -variances without further action-variances without further action
• Will show each type of variance is a statistical Will show each type of variance is a statistical answer to a different error question answer to a different error question And variance divergences are real indicators of And variance divergences are real indicators of
physical problems that can’t be glossed over physical problems that can’t be glossed over by swapping variancesby swapping variances
PSD LP(f) f p for p < 0
PTTI 2009 -- V. S. Reinhardt Page 3
Negative Power Law Noise,Reality vs Myth• Will also show it is Will also show it is mythmyth that one can properly that one can properly
separate polynomial deterministic behavior & separate polynomial deterministic behavior & (non-highpass filtered) neg-p noise(non-highpass filtered) neg-p noiseUsing least squares & Kalman filtersUsing least squares & Kalman filters
Except under certain conditionsExcept under certain conditions• Will show the Will show the realityreality is that such neg-p noise is is that such neg-p noise is
infinitely correlated & infinitely correlated & non-ergodicnon-ergodicEnsemble averages Ensemble averages time averages time averages
And this makes neg-p noise act more like And this makes neg-p noise act more like systematic error than conventional noise in systematic error than conventional noise in statistical estimationstatistical estimation
PTTI 2009 -- V. S. Reinhardt Page 4
tData Collection Interval T
= xc(tn) xr(t) = Noise + xr(tn)
xc(t) = True orDeterministic Behavior
x(tn) N Samples of Measured Data
Simple Statistical Estimation Model
Myth 1: Can “Fix” Variance Divergences Just by Swapping Variances
• Using a technique like a least squares or Kalman Using a technique like a least squares or Kalman filterfilter
• Note x(t) here is Note x(t) here is anyany data variable data variableNot necessarily the time errorNot necessarily the time error
Generate xa,M(t) an M-Parameter Est of xc(t)
PTTI 2009 -- V. S. Reinhardt Page 5
tTT
Basic Error Measures and the Questions They Address
• Can form variances from these error measuresCan form variances from these error measuresPoint Variance: Point Variance: (t)(t)22 = = EE xx(t)(t)22 (Kalman) (Kalman)EE .. = Ensemble average .. = Ensemble average Assumes Assumes EE xx(t) = 0(t) = 0
Average Variance: Average Variance: 22 = a weighted average of = a weighted average of
(t)(t)22 over T (LSQF) over T (LSQF)
Data Precision xj,M(tn): Data variation from fit?
xj,M(t)
Accuracy xw.M(tn): Error of fit from true behavior?
xw,M(t)
Mth Order -Measures ()Mx(t): Stability over ?
()2x(t)()x(t+)()x(t)=x(t+)-x(t)
PTTI 2009 -- V. S. Reinhardt Page 6
Interpreting -Measures as Mth Order Stability Measures
• Definition of MDefinition of Mthth Order Stability Order StabilityExtrapolationExtrapolation or or interpolationinterpolation error error xxj,Mj,M(t(tmm)) at an at an
(M+1)(M+1)thth point pointAfter a removing a perfect After a removing a perfect M-parameter fitM-parameter fit over over
only Monly M of those points of those points • Can show when Can show when xxa,Ma,M(t) = (M-1)(t) = (M-1)thth order polynomial order polynomial
& points separated by & points separated by xj,M(tm) ()Mx(t0) • Thus Thus -variances are statistical measures of -variances are statistical measures of
such stabilitysuch stability
xj,M(tm)xa,M(t)
Passes thru t0…tMbut not tm
xj,M(tM) xa,M(t)
Passes thru t0…tM-1
but not tM
x(tm)
Extrapolation Error Interpolation Error
PTTI 2009 -- V. S. Reinhardt Page 7
A Derived Error Measure: The Fit Precision (Error Bars)
• Fit Precision: A statistical estimate of accuracy Fit Precision: A statistical estimate of accuracy based on based on measuredmeasured data precision variance and data precision variance and correction factors based on a correction factors based on a specificspecific theoreticaltheoretical noise modelnoise model
wj,M(t)2 = d(t) j,M(t)2 wj,M2 = d j,M
2
• For uncorrelated noise & an unweighted linear For uncorrelated noise & an unweighted linear least squares fit (LSQF) least squares fit (LSQF) d = M/(N-M)Not true for correlated noiseNot true for correlated noise
txw,M(t)
xj,M(t)2wj,M(t) or 2wj,M
PTTI 2009 -- V. S. Reinhardt Page 8
The Neg-p Convergence Properties of Variances for Polynomial xa,M(t)
• For For -variances-variancesK(f) K(f) ff 2M2M (|f|<<1)(|f|<<1)
• Also true for Also true for MMthth order orderdata precisiondata precision
• Both converge for Both converge for 2M 2M -p -p• But for accuracy But for accuracy K(f)K(f)
is a lowpass filteris a lowpass filterAccuracy won’t converge unless Accuracy won’t converge unless |H|Hss(f)|(f)|22
provides sufficient HP filteringprovides sufficient HP filtering• Thus the temptation to “fix” a neg-p variance Thus the temptation to “fix” a neg-p variance
divergence by swapping variancesdivergence by swapping variances
K(f)|(f)H|(f)Ldfσ 2sp
2 ςSystem
ResponseVariance Kernel
PSD
-2 -1 0 1 2-150
-100
-50
0
1Log(fT)
M=5
f
10
M=4
f
8
M=3
f
6M=2 f
4M=1 f 2
dB
K(f) for Data Precision
1-M
0m
mmMa, ta(t)x
PTTI 2009 -- V. S. Reinhardt Page 9
This Is No “Fix” Because Each Variance Addresses a Different Error Question
• Arbitrarily swapping variances just misleadingly Arbitrarily swapping variances just misleadingly changes the question changes the question
• Does not remove the divergence problem for the Does not remove the divergence problem for the original questionoriginal question
tw,M(t)
j,M(t)2wj,M(t),M(t)
Accuracy: Error of fit from true behavior?Data Precision: Data variation from fit?Fit Precision: Est of accuracy from data/model?Stability: Extrap/Interp error over ?
PTTI 2009 -- V. S. Reinhardt Page 10
T
2w,12j,1
x(t) = f -2 Ensemble Membersxj,1
t0
Start Noise Here
Then What Does a Neg-p Variance Divergence Mean?
• Accuracy variance Accuracy variance as as tt00 while precision while precision remains remains misleadingly finitemisleadingly finite
• Thus an accuracy variance infinity is a real Thus an accuracy variance infinity is a real indicator of a severe modeling problemindicator of a severe modeling problemTo truly fix To truly fix Must modify the system design Must modify the system design
or the error question being askedor the error question being asked• Note for Note for ff -3 -3 noise noise j,1j,1 but but j,2j,2 remains finite remains finite• 1/f noise is a marginal case 1/f noise is a marginal case w,Mw,M
22 ln(f ln(fhhtt00))1/f contribution can be small even for t1/f contribution can be small even for t00 = age = age
of universeof universe
as t0
PTTI 2009 -- V. S. Reinhardt Page 11
Myth 2: Can Use Least Squares & Kalman Filters to Properly Separate• True polynomial behavior from data containing True polynomial behavior from data containing
(non-HP filtered) neg-p noise (non-HP filtered) neg-p noise • Will show that such noise acts like systematic Will show that such noise acts like systematic
error in statistical estimation error in statistical estimation Which generates anomalous fit resultsWhich generates anomalous fit resultsExcept under certain conditionsExcept under certain conditions
• Will show this occurs Will show this occurs Because (non-HP filtered) neg-p noise is Because (non-HP filtered) neg-p noise is
infinitely correlated & infinitely correlated & non-ergodicnon-ergodic
PTTI 2009 -- V. S. Reinhardt Page 12
NS & WSS Pictures of a Random Process xp(t)
• xxpp(t) starts at finite time(t) starts at finite time• Can have RCan have Rpp as t as tgg • Wigner-Ville functionWigner-Ville function
Is a tIs a tgg dependent PSD dependent PSD
• Two important NS Two important NS WSS WSS Theorems Theorems
• xxpp(t) active for all t(t) active for all t• xxpp(t) must have a (t) must have a boundedbounded
steady state steady state • PSD PSD
/2)(tx/2)(tx),(tR gpgpgp τττ E )(R)(tR pgp ττ ,
= Time Differencetg = Time from xp startE = Ensemble Average
),(tRLim)(R gpt
pg
ττ
),(tR f),(tW gpf,gp ττF
(Complex)Fourier Transform
τf,F
)(R (f)L pf,p ττF
f),(tWLim(f)L gpt
pg
NS Picturexp(t) = 0 t<0
0tg
WSS Picturexp(t) 0 all t
PTTI 2009 -- V. S. Reinhardt Page 13
The Properties of (Non-Highpass Filtered) Neg-p Noise• Rp(tg,) is finite for finite tg • Rp(tg,) as tg for all
• WSS WSS RRpp(() is infinite) is infinite for all for all
• Can defineCan define L Lpp(f)(f) without Rwithout Rpp(())
• Its correlation Its correlation time time cc is infinite is infinite
• It is inherently It is inherently non-ergodic non-ergodic Theorem (Parzan):Theorem (Parzan):
A NS random processA NS random processis ergodic if and only if is ergodic if and only if
f),(tWLim(f)L gpt
pg
Neg-p Noise
tg t=0
Unbounded as tg
Rp(tg,) < as tg and c <
),R(td,0)0.5R(tLim g
1-g
tc
g
τττ
E .. <..>T
),(tRLim)(R gpt
pg
ττ
PTTI 2009 -- V. S. Reinhardt Page 14
Ergodicity and Practical Fitting Behavior• Fitting Theory is Based on Fitting Theory is Based on EE• Practical fits rely on Practical fits rely on <..><..>TT • So noise must be So noise must be ergodic-likeergodic-like
over T over T <..>T E for a practical fit for a practical fitto work as theoretically predictedto work as theoretically predicted
• Even if noise is Even if noise is strictly ergodicstrictly ergodic<..>T = E May not have May not have <..>T E for the for the TT in question in question
• Most theory assumes Most theory assumes <..>any T = E for for N N CalledCalled local ergodicity local ergodicity for for TT00
• For correlated noise For correlated noise T >> c is also required is also requiredfor ergodic-like behavior (for ergodic-like behavior (intermediate ergodicityintermediate ergodicity))
Ensemble Av E
Time Av <..>T T
PTTI 2009 -- V. S. Reinhardt Page 15
• A single neg-p ensemble memberA single neg-p ensemble memberhas has polynomial-likepolynomial-like behavior over behavior over TTxxpp(t)(t) is is systematicsystematic withwith
polynomial-like xpolynomial-like xcc(t)(t)Even an augmented Kalman filter can only Even an augmented Kalman filter can only
separate non-systematic parts of xseparate non-systematic parts of xpp(t) & x(t) & xcc(t)(t)• Fitting methodologies can only separateFitting methodologies can only separate
linearly independentlinearly independent variables variables• Cannot truly separate (non-HP filtered) neg-p Cannot truly separate (non-HP filtered) neg-p
noise and poly-like deterministic behaviornoise and poly-like deterministic behaviorNoise whitening cannot be used in such casesNoise whitening cannot be used in such cases
f -2 Noise
f 0 Noise
Kalman Filter
f 0 Noise
LSQF
Practical Examples of Neg-p Non-ergodicity Fitting Effects
long term error-1.xls
AugmentedKalman
f -2 Noise
StandardKalman
f -3 Noise
xa,M ± wj,M
–x --xc
PTTI 2009 -- V. S. Reinhardt Page 16
Non-Ergodic-Like Behavior in Correlated WSS Processes
• T/T/cc = N = Nii Number of independent samples Number of independent samplesMust have NMust have Nii >> 1 for fit to be meaningful >> 1 for fit to be meaningful
• For non-HP filtered neg-p noise For non-HP filtered neg-p noise cc = = No No TT will produce ergodic-like fit behavior will produce ergodic-like fit behavior
- -True behavior
Kalman filter outputMeas dataFit precision
T
T/c=2000 T/c=200 T/c=20 T/c=2
PTTI 2009 -- V. S. Reinhardt Page 17
Conclusions
• Arbitrarily swapping variances does not really fix Arbitrarily swapping variances does not really fix a neg-p divergence problema neg-p divergence problemSuch divergences indicate real problems that Such divergences indicate real problems that
must be physically addressedmust be physically addressed• Non-HP filtered neg-p noise acts like more like Non-HP filtered neg-p noise acts like more like
systematic error than conventional noisesystematic error than conventional noiseWhen fitting to (full order) polynomials When fitting to (full order) polynomials The surest way to reduce such problems is to The surest way to reduce such problems is to
develop freq standards with lower neg-p noisedevelop freq standards with lower neg-p noiseGood news for frequency standards developersGood news for frequency standards developers
• See: See: http://http://www.ttcla.org/vsreinhardtwww.ttcla.org/vsreinhardt// for for preprints & other related materialpreprints & other related material