Case Studies in Time Series IICase Studies in Time Series IIPeriodic Behavior and Related issuesPeriodic Behavior and Related issues

David A. DickeyDavid A. Dickey

Professor of StatisticsProfessor of Statistics

N. C. State UniversityN. C. State University

An Example - RollersAn Example - Rollers

circumferences 11, 8, 5, 16, 6,circumferences 11, 8, 5, 16, 6, 20 inches 20 inches

Rollers are to even out fabric coating Rollers are to even out fabric coating Offset roller => Periodicity in thicknessOffset roller => Periodicity in thickness Period = Roller circumferencePeriod = Roller circumference YYtt = thickness at time = thickness at time ARIMA model does not help!ARIMA model does not help!

““Frequency Domain”Frequency Domain”

Coating Thickness every ¼ inchCoating Thickness every ¼ inch

PeriodicityPeriodicity ??????

One “radian”One “radian”

Fun with radians Fun with radians Half circle is Half circle is radians radians Assume all angles in [ - Assume all angles in [ - Rotation slides along sine waveRotation slides along sine wave

-1 x

Fun with trigonometry Fun with trigonometry

A Sin(A Sin(t+t+) = A Sin() = A Sin(t)Cos(t)Cos( A Cos( A Cos(t)Sin(t)Sin())

Got Got Create Sin(Create Sin(t), Cos(t), Cos(t)t)

[A Cos([A Cos(Sin(Sin(t)t) [A Sin( [A Sin()] Cos()] Cos(t)t)

Sin(Sin(t)t) Cos( Cos(t)t)

Regression !!!Regression !!!

-- got -- got

[A Cos([A Cos(Sin(Sin(t)t) [A Sin( [A Sin()] Cos()] Cos(t)t) Sin(Sin(t)t) Cos( Cos(t)t) 22 = A = A22 [Cos [Cos22(( Sin Sin22((

Tan(Tan()) = Arctan(= Arctan() ) From regression – recoverFrom regression – recover

Amplitude AAmplitude APhase angle Phase angle

Harmonics (wave Harmonics (wave formform) 2) 2

Intcpt sine cos sine cos sine cos 1 0.17 0.98 0.34 0.94 0.50 0.87 1 0.34 0.94 0.64 0.77 0.87 0.50 1 0.50 0.87 0.87 0.50 1.00 0.00 1 0.64 0.77 0.98 0.17 0.87 -.50 1 0.77 0.64 0.98 -.17 0.50 -.87 1 0.87 0.50 0.87 -.50 0.00 -1.0 1 0.94 0.34 0.64 -.77 -.50 -.87 1 0.98 0.17 0.34 -.94 -.87 -.50 1 1.00 0.00 0.00 -1.0 -1.0 -.00 1 0.98 -.17 -.34 -.94 -.87 0.50 1 0.94 -.34 -.64 -.77 -.50 0.87 1 0.87 -.50 -.87 -.50 -.00 1.00 1 0.77 -.64 -.98 -.17 0.50 0.87 1 0.64 -.77 -.98 0.17 0.87 0.50

(more values)

1 -.17 0.98 -.34 0.94 -.50 0.87 1 -.00 1.00 -.00 1.00 -.00 1.00

X’X = (n/2)I (mostly) X’X = (n/2)I (mostly)

n/2(n/2(2)2) = (n/2)A = (n/2)A22 = regression SSq = regression SSq

jjjjn = jn = jthth “Fourier Frequency” “Fourier Frequency”

( (n-1)/2 of these if n odd)( (n-1)/2 of these if n odd) PROC SPECTRAPROC SPECTRA

Calculate all Fourier Sin, Cos (n-1)Calculate all Fourier Sin, Cos (n-1)Regress on all (multiple regression)Regress on all (multiple regression)

orthogonal!!!orthogonal!!! Type I = Type III = “Periodogram Ordinate”Type I = Type III = “Periodogram Ordinate”

PolynomialsPolynomials

Enough XEnough Xjj terms (n-1) => fit data perfectly terms (n-1) => fit data perfectly

““Taylor Series Expansion”Taylor Series Expansion”

Goal: Enough, not too manyGoal: Enough, not too many

Periodogram Periodogram

Enough terms (n-1)/2 sines, (n-1)/2 cosines =>Enough terms (n-1)/2 sines, (n-1)/2 cosines =>

fit data perfectly!fit data perfectly!

““Fourier Series Expansion” Fourier Series Expansion”

Goal: Enough, not too many Goal: Enough, not too many

Periodogram AnalysisPeriodogram Analysis Plot Periodogram ordinate Plot Periodogram ordinate jj) vs. ) vs. jj

Look for PeriodicitiesLook for Periodicities

Our case – can just do periodicities of rollers.Our case – can just do periodicities of rollers.

Spectral density = smoothed, normalized Spectral density = smoothed, normalized periodogramperiodogram

proc spectra data=fabric p s adjmean out=outspec; var Y;

weights 1 2 3 4 5 4 3 2 1;

proc gplot data=outspec;

plot P_01*freq S_01*freq;

symbol1 v=diamond i=needle c=green;

Adjmean: set mean to 0 ( I(0) = n Ybar2) P periodogram S smoothed periodogram

Moving average smoother weights

Important Frequencies

proc print data=outspec; where P_01 > 20proc print data=outspec; where P_01 > 20 Obs FREQ PERIOD P_01 S_01Obs FREQ PERIOD P_01 S_01

17 0.19635 32.0000 47.8072 0.9274317 0.19635 32.0000 47.8072 0.92743 27 0.31907 19.6923 51.6533 1.1792227 0.31907 19.6923 51.6533 1.17922

Two of our rollers are likely out of line. Two of our rollers are likely out of line. circumferences 5, 6, 8, 11, 16 and 20 inches circumferences 5, 6, 8, 11, 16 and 20 inches

One roller that is out of line would rotate once in 32 One roller that is out of line would rotate once in 32 observations. With 4 observations per inch, that roller observations. With 4 observations per inch, that roller would have circumference 8 inches. would have circumference 8 inches.

A roller of circumference 19.69 quarter inches (4.9 A roller of circumference 19.69 quarter inches (4.9 inches) is not one of our known circumferences. Perhaps inches) is not one of our known circumferences. Perhaps it is really the 5 inch one.it is really the 5 inch one.

Important Frequencies ??

data waves; pi=4*atan(1); data waves; pi=4*atan(1); set fabric; set fabric; s5 = sin(2*pi*location/5); c5 = cos(2*pi*location/5);s5 = sin(2*pi*location/5); c5 = cos(2*pi*location/5); s6 = sin(2*pi*location/6); c6 = cos(2*pi*location/6);s6 = sin(2*pi*location/6); c6 = cos(2*pi*location/6); s8 = sin(2*pi*location/8); c8 = cos(2*pi*location/8);s8 = sin(2*pi*location/8); c8 = cos(2*pi*location/8); s11 = sin(2*pi*location/11); c11 = cos(2*pi*location/11);s11 = sin(2*pi*location/11); c11 = cos(2*pi*location/11); s16 = sin(2*pi*location/16); c16 = cos(2*pi*location/16);s16 = sin(2*pi*location/16); c16 = cos(2*pi*location/16); s20 = sin(2*pi*location/20); c20 = cos(2*pi*location/20);s20 = sin(2*pi*location/20); c20 = cos(2*pi*location/20); proc reg; model Y = s5--c20/ss1 ss2; proc reg; model Y = s5--c20/ss1 ss2; T_5: test s5=0, c5=0; T_6: test s6=0, c6=0; T_5: test s5=0, c5=0; T_6: test s6=0, c6=0; T_8: test s8=0, c8=0; T_11: test s11=0, c11=0; T_8: test s8=0, c8=0; T_11: test s11=0, c11=0; T_16: test s16=0, c16=0; T_20: test s20=0, c20=0; T_16: test s16=0, c16=0; T_20: test s20=0, c20=0;

The TEST statement results areThe TEST statement results are

5 inch5 inch 6 inch 6 inch 8 inch8 inch 37.0 37.0 0.22 0.22 20.0820.08 Pr > F = Pr > F = <.0001<.0001 .8018.8018 <.0001<.0001

11 inch 16 inch 20 inch11 inch 16 inch 20 inch 3.27 2.72 1.363.27 2.72 1.36 Pr > F = .0387 .0670 .2586Pr > F = .0387 .0670 .2586

Theoretical Spectral DensityTheoretical Spectral Density( ( (h) = cov{Y(h) = cov{Ytt, Y, Yt-ht-h} )} )

h

hhf )cos()(2

1)(

spectrumAR(1) ))) cos( 2- (1/(2

2)(

22

2

spectrumnoisewhitef

Generic

2 specific cases

Generate Theoretical DensitiesGenerate Theoretical Densities Data AR_1; Data AR_1; do t = 1 to 512; e= normal(123); Y = .8*Y + 2*e ;do t = 1 to 512; e= normal(123); Y = .8*Y + 2*e ; if Y=. then Y=0; X = -.8*X + 2*e; if X=. then X=0; drop e;if Y=. then Y=0; X = -.8*X + 2*e; if X=. then X=0; drop e; output; end; output; end; proc spectra P S adjmean out=arspec; Var Y X; proc spectra P S adjmean out=arspec; Var Y X; weight 1 2 3 4 5 6 7 6 5 4 3 2 1; weight 1 2 3 4 5 6 7 6 5 4 3 2 1; data arspec; set arspec; data arspec; set arspec; Ytheory = 4/(2*3.14159*(1.64 - 1.6*cos(freq))); Ytheory = 4/(2*3.14159*(1.64 - 1.6*cos(freq))); Xtheory = 4/(2*3.14159*(1.64 + 1.6*cos(freq)));Xtheory = 4/(2*3.14159*(1.64 + 1.6*cos(freq))); label S_01 = "Y (.8)"; label S_02 = "X (-.8)"; label S_01 = "Y (.8)"; label S_02 = "X (-.8)"; proc gplot; plot (S_01 Ytheory)*freq/overlay; proc gplot; plot (S_01 Ytheory)*freq/overlay; symbol1 v=dot i=none c=red h=.6; symbol1 v=dot i=none c=red h=.6; symbol2 v=none i=join c=black w=3; symbol2 v=none i=join c=black w=3; plot (S_02 Xtheory)*freq/overlay; run;plot (S_02 Xtheory)*freq/overlay; run;

Overlay Theoretical & Overlay Theoretical & EstimatedDensitiesEstimatedDensities

Autocorrelation > 0 Autocorrelation < 0

Plant Enzyme ActivityPlant Enzyme Activity

Significant ?

proc spectra whitetest; proc spectra whitetest; The SPECTRA ProcedureThe SPECTRA Procedure Test for White Noise for Variable YTest for White Noise for Variable Y M-1 14M-1 14 Max(P(*)) 11932.89Max(P(*)) 11932.89 Sum(P(*)) 34590.36Sum(P(*)) 34590.36 Fisher's Kappa: (M-1)*Max(P(*))/Sum(P(*))Fisher's Kappa: (M-1)*Max(P(*))/Sum(P(*)) Kappa 4.829682Kappa 4.829682 Bartlett's Kolmogorov-Smirnov Statistic:Bartlett's Kolmogorov-Smirnov Statistic: Maximum absolute difference of the standardizedMaximum absolute difference of the standardized partial sums of the periodogram and the CDF of apartial sums of the periodogram and the CDF of a uniform(0,1) random variable.uniform(0,1) random variable. Test Statistic 0.255984Test Statistic 0.255984 Approximate P-Value 0.3180Approximate P-Value 0.3180

4.877 at the 5% level4.877 at the 5% level (e.g. Fuller 1996)(e.g. Fuller 1996)

Lesson learned Lesson learned

F test F test waswas significant (regression) significant (regression) Spectral tests not significant.Spectral tests not significant.

Price paid (power) for not knowing Price paid (power) for not knowing

frequency !!frequency !!

Researcher Researcher expectedexpected this frequency in this frequency in

advance.advance.

ln {NC Retail Sales (million $)}ln {NC Retail Sales (million $)}

periodicity

\ }- trend/

Periodogram

Boxes: residuals from broken trend

Residual Periodogram Residual Periodogram & PROC GAM fit& PROC GAM fitModel:

Tend & break & seasonal dummies

Low frequency activity

AUTOREG Model: Tend & break & seasonal dummies

Flat => “white noise” ( et )

Bartlett Test Statistic 0.153536 Approximate P-Value 0.0671

Transfer Functions X-> YTransfer Functions X-> Y YYtt = __ + __X = __ + __Xtt + __X + __Xt-1t-1 +__X +__Xt-2t-2 + … + e + … + ett

Recall: Recall:

Xt has Fourier ExpansionXt has Fourier Expansion

Yt has Fourier ExpansionYt has Fourier Expansion

X sinusoid -> Y sinusoid at X sinusoid -> Y sinusoid at same frequencysame frequency

Amplitude (A) may changeAmplitude (A) may change

Phase shift (Phase shift () )

Neuse River flows – up & down streamNeuse River flows – up & down stream

Phase Coherency (R2)

More or less smoothing, & model basedNote: phase slope gives delay in Yt = __Xt-d

Gain = amplitude changeGain = amplitude change

All stats available from PROC SPECTRA ( CROSS options ). Learn more: Fuller (1996) Brocklebank & Dickey (2003)

Thanks !Thanks !Questions ?Questions ?