Frequency Domain Techniques i n Forecasting with ARIMA Model

Post on 23-Feb-2016

84 views 2 download

Tags:

description

Frequency Domain Techniques i n Forecasting with ARIMA Model. Dr. M. J ahanur Rahman Associate Professor Department of Statistics University of Rajshahi , Rajshahi-6205, Bangladesh Jahan.stat@ru.ac.bd July 28, 2012. Contents. Time Series? Time domain forecasting with ARIMA model - PowerPoint PPT Presentation

Transcript of Frequency Domain Techniques i n Forecasting with ARIMA Model

Frequency Domain Techniques in

Forecasting with ARIMA Model

Dr. M. Jahanur RahmanAssociate Professor

Department of StatisticsUniversity of Rajshahi, Rajshahi-6205, Bangladesh

Jahan.stat@ru.ac.bd

July 28, 2012

Dr. M. Jahanur Rahman, R.U.

Contents

Time Series?

Time domain forecasting with ARIMA model

Transformations from time domain to

frequency domain

Frequency domain techniques in forecasting

with ARIMA model

2

Dr. M. Jahanur Rahman, R.U. 3

Time Series

A sequence of observations indexed by time, denoted

by or

Monthly relative humidity time series of Rajshahi

Dr. M. Jahanur Rahman, R.U. 4

𝑦 𝑡+2

𝑦 𝑡+1𝑦 𝑡

𝑦 𝑡−1

𝑦 𝑡−2

= Realization

orSample pathTime Series?

A finite realization of a time series process

Let be a sequence of random variable indexed by time t.

Random Process to Time Series

Dr. M. Jahanur Rahman, R.U. 5

Note that

Time series is only one of the many possible

realizations (sample paths) that the history might

have generated.

Time series analysis is trying to draw statistical

inference from a single outcome (time series),

even partially observed.

Dr. M. Jahanur Rahman, R.U. 6

How can we make sensible

inferences on the underlying

time series process with a single

observation?We need a strong assumption:

Stationarity (or ergodicity). This assumption treat the time series as a random sample

from the same underlying population (process).

Dr. M. Jahanur Rahman, R.U. 7

A time series is weak stationary if its mean, variance, and autocovariance (at various lags) remain the same no matter at what point of time we measure them. That is, they are time invariant.

Stationary Time Series

Mea

n =

Varia

nce

= Co

varia

nce

=

Mea

n =

Varia

nce

= Co

varia

nce

=

Mea

n =

Varia

nce

= Co

varia

nce

=

Mea

n =

Varia

nce

= Co

varia

nce

=

Mea

n =

Varia

nce

= Co

varia

nce

=

Dr. M. Jahanur Rahman, R.U. 8

Ergodicity

a realization(a sample path)

Mea

n =

Varia

nce

= Co

varia

nce

=

Mea

n =

Varia

nce

= Co

varia

nce

=

Mea

n =

Varia

nce

= Co

varia

nce

=

Mea

n =

Varia

nce

= Co

varia

nce

=

Mea

n =

Varia

nce

= Co

varia

nce

=

Dr. M. Jahanur Rahman, R.U.

Mea

n =

Varia

nce

= Co

varia

nce

=

Mea

n =

Varia

nce

= Co

varia

nce

=

Mea

n =

Varia

nce

= Co

varia

nce

=

Mea

n =

Varia

nce

= Co

varia

nce

=

Mea

n =

Varia

nce

= Co

varia

nce

=

Covariance = Variance =

Mean =

The covariance-stationary process is ergodic for the mean.

The covariance-stationary process is ergodic for the variance.

The covariance-stationary process is ergodic for the covariance.9

Ergodicity

Dr. M. Jahanur Rahman, R.U. 10

Non-stationary time series Non-stationary time series

-3

-2

-1

0

1

2

3

10 20 30 40 50 60 70 80 90 00

Stationary time series

(Non)Stationay Time Series

Dr. M. Jahanur Rahman, R.U. 11

Non-stationary time series contain two kinds of trends: Deterministic trend Stochastic (random) trend

Non-stationary Time Series

Dr. M. Jahanur Rahman, R.U. 12

Trend Stationary: If y(t) contains a deterministic trend and

{y(t) – trend}

becomes stationary. Then y(t) is known as trend stationary.

Difference Stationary: If y(t) contains a stochastic trend and

{y(t) – y(t-1)}

becomes stationary. Then y(t) is known as difference stationary.

Non-stationary Time Series

Dr. M. Jahanur Rahman, R.U. 13

Non-stationary Time Series

Dr. M. Jahanur Rahman, R.U. 14

Any time series can contain some or all of the following

components:

1. Trend (T)

2. Cyclical (C)

3. Seasonal (S)

4. Irregular (I)

Trend component

The trend is the long term pattern of a time series. A trend can be

positive or negative depending on whether the time series exhibits an

increasing long term pattern or a decreasing long term pattern.

Components of a Time Series

Dr. M. Jahanur Rahman, R.U. 15

Cyclical component

Any pattern showing an up and down movement around a given trend

is identified as a cyclical pattern. The duration of a cycle depends on

the type of business or industry being analyzed.

Any time series can contain some or all of the following

components:

1. Trend (T)

2. Cyclical (C)

3. Seasonal (S)

4. Irregular (I)

Components of a Time Series

Seasonal component

Seasonality occurs when the time series exhibits regular fluctuations

during the same month (or months) every year, or during the same

quarter every year. For instance, retail sales peak during the month of

December.

Any time series can contain some or all of the following

components:

1. Trend (T)

2. Cyclical (C)

3. Seasonal (S)

4. Irregular (I)

Components of a Time Series

Dr. M. Jahanur Rahman, R.U. 16

Dr. M. Jahanur Rahman, R.U. 17

Irregular component

This component is unpredictable. Every time series has some

unpredictable component that makes it a random variable. In prediction,

the objective is to “model" all the components to the point that the only

component that remains unexplained is the random component.

Any time series can contain some or all of the following

components:

1. Trend (T)

2. Cyclical (C)

3. Seasonal (S)

4. Irregular (I)

Components of a Time Series

Dr. M. Jahanur Rahman, R.U. 18

Components of a Time Series

Dr. M. Jahanur Rahman, R.U. 19

Two DomainsTraditionally, there are two ways to analyze time series data:

Time domain analysis

Time domain analysis examines how a time series process evolves

through time, with tools such as autocorrelation function.

Frequency domain analysis

Frequency domain analysis, also known as spectral analysis,

studies how periodic components at different frequencies

describe the evolution of a time series.

Dr. M. Jahanur Rahman, R.U. 20

Time

Time domainFrequency

Frequency domain

Two Domains

Dr. M. Jahanur Rahman, R.U. 21

The Autoregressive Integrated Moving Average (ARIMA)

models, or Box-Jenkins models, are a class of linear models

that is capable of representing stationary as well as non-

stationary time series.

Time Series Model

Dr. M. Jahanur Rahman, R.U. 22

ARIMA(p,d,q) Model

∆𝑑𝑌 𝑡=𝑐+𝜙1∆𝑑𝑌 𝑡− 1+𝜙2∆

𝑑𝑌 𝑡 −2+⋯𝜙𝑝 ∆𝑑𝑌 𝑡−𝑝+𝜀𝑡+𝜃1𝜀𝑡 −1+𝜃2𝜀𝑡− 2+⋯+𝜃𝑞𝜀𝑡 −𝑞

= Response (dependent) variable at time

= Respose variable at time lags

= difference operator ()

= White noise error term

= Errors in previous time periods

= parameters

= Number of autoregressive terms

= Number of moving average terms.

Dr. M. Jahanur Rahman, R.U. 23

ARIMA(p,0,q) = ARMA(p, q) Model

ARIMA(p,0,0) = AR(p) Model

ARIMA(0,0,q) = MA(q) Model

Dr. M. Jahanur Rahman, R.U. 24

ARIMA models rely heavily on both autocorrelation (AC) and

partial autocorrelation (PAC) patterns in time series data.

Dr. M. Jahanur Rahman, R.U. 25

ARIMA(2,0,0) = AR(2)

AC PA

C

1 2 3 4 5 6 7 …….. Lags 1 2 3 4 5 6 7 …….. Lags

ARIMA(0,0,2) = MA(2)1 2 3 4 5 6 7 …….. Lags 1 2 3 4 5 6 7 …….. Lags

AC

PAC

ARIMA(2,0,2) = ARMA(2, 2)

AC

PAC

1 2 3 4 5 6 7 …….. Lags1 2 3 4 5 6 7 …….. Lags

Theoretical pattern of ACF and PACFACF 0PACF = 0 for lag > 2

ACF = 0 for lag > 2; PACF 0

ACF PACF 0

Dr. M. Jahanur Rahman, R.U. 26

Choosing an ARIMA model

ARIMA modeling requires a great deal of skill, which of course come from practice.

Model Pattern of ACF Pattern of PACFAR(p) Spikes decays exponentially. Significance spikes through lags p

MA(q) Significance spikes through lags q Spikes decays exponentially

ARMA(p,q) Spikes decays exponentially Spikes decays exponentially

Theoretical pattern of ACF and PACF

Dr. M. Jahanur Rahman, R.U. 27

1. Identification of the model

(choosing tentative p, d, p for ARIMA)

2. Parameter estimation of the chosen model.

4. Forecasting

3. Diagnostic checking

(are the estimated residual white noise?)

yes

no

Box-Jenkins (BJ) Methodology

Dr. M. Jahanur Rahman, R.U. 28

Steps of modeling a observed time series

with ARIMA

Dr. M. Jahanur Rahman, R.U. 29

Original time series [

Log transformation

Dr. M. Jahanur Rahman, R.U. 30Regular differencing

Seasonal differencing

Dr. M. Jahanur Rahman, R.U. 31

Correlogram: Shows SAC & SPAC at different lags

Dr. M. Jahanur Rahman, R.U. 32

ARIMA(2,0,0) = AR(2)

AC PA

C

1 2 3 4 5 6 7 …….. Lags

1 2 3 4 5 6 7 ……..

ARIMA(0,0,2) = MA(2)1 2 3 4 5 6 7 …….. Lags

1 2 3 4 5 6 7 ……..

AC

PAC

ARIMA(2,0,2) = ARMA(2, 2)

AC

PAC

1 2 3 4 5 6 7 ……1 2 3 4 5 6 7 …….. Lags

Competitive Models:

(1) ARMA(2, 0) (2) ARMA(0, 1) (3) ARMA(2, 1)

Choosing an ARIMA model

Dr. M. Jahanur Rahman, R.U. 33

Model MSE

ARMA(2, 0) 3521

ARMA(0, 1) 4019

ARMA(2, 1) 3998

Looking for the best Model

Dr. M. Jahanur Rahman, R.U. 34

Finally Forecasting

Frequency Domain Fourier Transformation Short-time Fourier Transformation Wavelet Transformation

Dr. M. Jahanur Rahman, R.U. 36

Frequency Domain

To get further information from the time series that is not

readily available in the time domain.

Frequency Domain

Time Domain

Dr. M. Jahanur Rahman, R.U. 37

Autocorrelation function or correlogram is used for analyzing time series in time domain.

rk

k

Correlogram

t

Yt

Periodic process with noise

Time Domain

Periodicities in data can be best determined by analyzing the time series in frequency domain.

Time series

Dr. M. Jahanur Rahman, R.U. 38

Frequency Domain

Frequency Domain

Time Domain

Transformations:

Fourier Transformation (FT)

Short-time Fourier Transformation (SFT)

Wavelet Transformation (WT) ………

Dr. M. Jahanur Rahman, R.U. 39

Time series analysis in frequency domain is known as

frequency domain analysis or spectral analysis.

Dr. M. Jahanur Rahman, R.U. 40

The Basics

= Amplitude (height of the function), R > 0

= Frequency (cycle/unit time)

= Phase (the starting point of the cosine function

which lies in )

(period).

𝑌 𝑡=𝑅𝑐𝑜𝑠 (2𝜋 𝑓𝑡+𝜙 )

Dr. M. Jahanur Rahman, R.U. 41

𝑌 𝑡=𝑅𝑐𝑜𝑠 (2𝜋 𝑓𝑡+𝜙 )

t = running from 1 to 100. Red curve has , = 4/100 = 1/25 and Blue curve has , = 4/100 = 1/25 and Black curve has , = 4/100 = 1/25 and

The Basics

Dr. M. Jahanur Rahman, R.U. 42

as

𝑌 𝑡=𝑅𝑐𝑜𝑠 (2𝜋 𝑓𝑡+𝜙 )

The Basics

Dr. M. Jahanur Rahman, R.U. 43

𝑦 2𝑡=2 cos (2𝜋 5 𝑓𝑡 )+3sin (2𝜋 5 𝑓𝑡 )

𝑦 1𝑡=2cos (2𝜋 2 𝑓𝑡 )+3sin (2𝜋 2 𝑓𝑡 )

𝑦 𝑡=𝑦1 𝑡+𝑦2 𝑡

Jean Baptiste Joseph Fourier(1768-1830)

The Basics

Dr. M. Jahanur Rahman, R.U. 44

Mat

hem

atic

al C

once

pts

Dr. M. Jahanur Rahman, R.U. 45

𝑓 (𝑥 )= ⟨ 𝑓 (𝑥) ,1 ⟩⟨1,1 ⟩

1+∑𝑛=1

∞ ⟨ 𝑓 (𝑥) ,cos (𝑛𝑥) ⟩⟨cos (𝑛𝑥 ) ,cos (𝑛𝑥) ⟩

cos (𝑛𝑥)+∑𝑛=1

∞ ⟨ 𝑓 (𝑥) , sin (𝑛𝑥 )⟩⟨sin (𝑛𝑥 ), sin (𝑛𝑥 )⟩

sin (𝑛𝑥 )

So 𝑎0=1

2𝜋 ∫−𝜋

𝜋

𝑓 (𝑥 )𝑑𝑥

𝑎𝑛=1𝜋 ∫

−𝜋

𝜋

𝑓 (𝑥 )cos (𝑛𝑥 )𝑑𝑥

𝑏𝑛=1𝜋 ∫

−𝜋

𝜋

𝑓 (𝑥 )𝑠𝑖𝑛 (𝑛𝑥 )𝑑𝑥 𝑛=1,2,3 ,……

𝑓 (𝑥 )=𝑎0+∑𝑛=1

𝑎𝑛cos (𝑛𝑥 )+∑𝑛=1

𝑏𝑛sin (𝑛𝑥)

According to Fourier, any function in can be defined as

Fourier Series

Dr. M. Jahanur Rahman, R.U. 46

or equivalently,

Where

for

Fourier Series

Dr. M. Jahanur Rahman, R.U. 47

Time (t)

y(t)

Frequency (f)

Y(f)

Time domain to Frequency Domain

Time Domain Frequency Domain

Dr. M. Jahanur Rahman, R.U. 48

𝑦 2𝑡=2 cos (2𝜋 5 𝑓𝑡 )+3sin (2𝜋 5 𝑓𝑡 )

𝑦 1𝑡=2cos (2𝜋 2 𝑓𝑡 )+3sin (2𝜋 2 𝑓𝑡 )

𝑦 𝑡=𝑦1 𝑡+𝑦2 𝑡

2f 5f

5f

2f

Time Domain Frequency Domain

Time domain to Frequency Domain

Dr. M. Jahanur Rahman, R.U. 49

Discrete Fourier Transformation (DFT)Let a time sequence

and a frequency sequence

The DFT of is

Where

Note that and are of the same size.

The DFT of a time series is another time series.

Dr. M. Jahanur Rahman, R.U. 50

The inverse DFT (IDET) of , is given by

Inverse Discrete Fourier Transformation (IDFT)

DFT

IDFT

3f 40f6f

Fourier Transformation

Inverse Fourier Transformation.

Dr. M. Jahanur Rahman, R.U. 51

Spectral Density

Spectral density () is the amount of variance per interval of frequency: Angular frequency:

The plot of vs is called spectrum and as a whole called spectral density.

Dr. M. Jahanur Rahman, R.U. 52

Spectral Density

𝐼 (𝑘 )

𝜔𝑘

Spectral density transforms information from time domain to frequency domain

Spectral density

Dr. M. Jahanur Rahman, R.U. 53

Total area under the spectrum is equal to the variance of the process.

All information in frequency domain is extracted from the spectral density (or spectrum).

Spectral density () is the amount of variance per interval of frequency.A peak in the spectrum indicates an important contribution to

the variance at frequencies close the peak.Prominent spikes indicate periodicity. Several expressions for spectrum exist in literature.

Spectral Density

Dr. M. Jahanur Rahman, R.U. 54

Spectral Density For a complete random series the spectral density function is

constant – termed as white noise.White noise indicates that no frequency interval contains

more variance than any other frequency intervals.

𝐼 (𝑘 )

𝜔𝑘Spectral density of a White noise

Dr. M. Jahanur Rahman, R.U. 55

3f 40f6f

DFT

Time Series Periodogram

IDFT

Filtered Series

Remove high

frequency component

ReconstructionDecomposition

Application of Fourier Transformation

Dr. M. Jahanur Rahman, R.U. 56

0 0.2 0.4 0.6 0.8 1-3

-2

-1

0

1

2

3

0 5 10 15 20 250

100

200

300

400

500

600

All frequency components exist at all times

Stationary Time Series

Fourier Transformation

Dr. M. Jahanur Rahman, R.U. 57

Frequency

Am

plitu

de

Fourier TransformationNon-stationary Series

Dr. M. Jahanur Rahman, R.U. 58

0 0.5 1-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20 250

50

100

150

Time Frequency0 0.5 1

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20 250

50

100

150

Time Frequency

Different in Time Domain

Same in Frequency Domain

FT only gives frequency components exist in the seriesThe time and frequency Information can not be seen at the same

time which is required.

Fourier TransformationNon-stationary Series

Dr. M. Jahanur Rahman, R.U. 59

Short-Time Fourier TransformationDennis Gabor (1946) In STFT, the signal is divided into small

enough segments, where these segments of the series can be assumed to be stationary.

For this purpose, a fixed window function "w" is chosen

Dr. M. Jahanur Rahman, R.U. 60

Short-Time Fourier TransformationDennis Gabor (1946) In STFT, the signal is divided into small

enough segments, where these segments of the series can be assumed to be stationary.

For this purpose, a fixed window function "w" is chosen

Dr. M. Jahanur Rahman, R.U. 61

FT X

Short-Time Fourier Transformation

Dr. M. Jahanur Rahman, R.U. 62

FT X

Short-Time Fourier Transformation

Dr. M. Jahanur Rahman, R.U. 63

FT X

Advantage & Disadvantage

Short-Time Fourier Transformation

Dr. M. Jahanur Rahman, R.U. 64

FT X

Short-Time Fourier Transformation

Dr. M. Jahanur Rahman, R.U. 65

FT X

Short-Time Fourier Transformation

Dr. M. Jahanur Rahman, R.U. 66

Time stepFrequency

Am

plitu

deShort-Time Fourier Transformation

Dr. M. Jahanur Rahman, R.U. 67

Time

Freq

uenc

y

Better time resolution; Poor frequency resolution

Better frequency resolution; Poor time resolution

Narrow windows give good time resolution, but poor frequency resolution.

Wide windows give good frequency resolution, but poor time resolution.

Advantage & Disadvantage

Short-Time Fourier Transformation

Dr. M. Jahanur Rahman, R.U. 68

Wide windows do not provide good localization at high frequencies

Advantage & Disadvantage

Short-Time Fourier Transformation

Dr. M. Jahanur Rahman, R.U. 69

Use narrower windows at high frequencies

Advantage & Disadvantage

Short-Time Fourier Transformation

Dr. M. Jahanur Rahman, R.U. 70

Narrow windows do not provide good localization at low frequencies

Advantage & Disadvantage

Short-Time Fourier Transformation

Dr. M. Jahanur Rahman, R.U. 71

Use wider windows at low frequencies

Advantage & Disadvantage

Short-Time Fourier Transformation

Dr. M. Jahanur Rahman, R.U. 72

= Small value

(Wavelet coefficient)

.54321

TimeSc

ale

Scale = 1, time = 0

Changing Window at Multi-resolution Analysis

Dr. M. Jahanur Rahman, R.U. 73

= Small value

(Wavelet coefficient)

.54321

TimeSc

ale

Scale = 1, time = 100

Changing Window at Multi-resolution Analysis

Dr. M. Jahanur Rahman, R.U. 74

= Moderate value

(Wavelet coefficient)

.54321

TimeSc

ale

Scale = 1, time = 500

Changing Window at Multi-resolution Analysis

Dr. M. Jahanur Rahman, R.U. 75

= Large value

(Wavelet coefficient)

.54321

TimeSc

ale

Scale = 1, time = 800

Changing Window at Multi-resolution Analysis

Dr. M. Jahanur Rahman, R.U. 76

Mag

nitu

de

20 Hz 50 Hz 120 Hz

Changing Window at Multi-resolution Analysis

Dr. M. Jahanur Rahman, R.U. 77

The Discrete Wavelet Transform (DWT)

𝐷𝑊𝑇 (𝑚 ,𝑛 )=∑𝑡𝑌 (𝑡 )Ψ (𝑚 ,𝑛 )

= wavelet function = mother wavelet m = is location parameter n = scale parameter = time series

Dr. M. Jahanur Rahman, R.U. 78

Wavelet families

Dr. M. Jahanur Rahman, R.U. 79

Multi-level Decomposition of a Series with Wavelets

Time Series

cA3 cD3

cA2 cD2

cA0 cD0The decomposition tree can be schematically described as:

Dr. M. Jahanur Rahman, R.U. 80

0 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 0 - 0 0 0 0 0 00 0 - 0 0 0 00 0 0 0 - 0 00 0 0 0 0 0 -

0 0 0 00 0 0 0

- - 0 0 0 00 0 0 - -

- - - -

Basis Constriction

Dr. M. Jahanur Rahman, R.U. 81

Example

Dr. M. Jahanur Rahman, R.U. 82

Multi-level Decomposition of a Series with Wavelets

Time Series1, 3, 5, 11, 12, 13, 0, 1

cA32.8, 11.3, 17.6, 0.7

cD3-1.4, -4.2, -0.7, -0.7

0 0 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 0 - 0 0 0 0 0 00 0 - 0 0 0 00 0 0 0 - 0 00 0 0 0 0 0 -

Level-1 orScale-1

Dr. M. Jahanur Rahman, R.U. 83

Multi-level Decomposition of a Series with Wavelets

Time Series1, 3, 5, 11, 12, 13, 0, 1

cA32.8, 11.3, 17.6, 0.7

cD3-1.4, -4.2, -0.7, -0.7

Level-1 orScale-1

Wavelet coefficients = {2.8, 11.3, 17.6, 0.7, -1.4, -4.2, -0.7, -0.7}

Dr. M. Jahanur Rahman, R.U. 84

Multi-level Decomposition of a Series with Wavelets

Time Series1, 3, 5, 11, 12, 13, 0, 1

cA32.8, 11.3, 17.6, 0.7

cD3-1.4, -4.2, -0.7, -0.7

cA210.0, 13.0

cD2-6.0, 12.0

Level-1 orScale-1

Level-2 orScale-2

0 0 0 00 0 0 0

- - 0 0 0 00 0 0 - -

Wavelet coefficients = {10.0, 13.0, -6.0, 12.0, -1.4, -4.2, -0.7, -0.7}

Dr. M. Jahanur Rahman, R.U. 85

Multi-level Decomposition of a Series with Wavelets

Time Series1, 3, 5, 11, 12, 13, 0, 1

cA32.8, 11.3, 17.6, 0.7

cD3-1.4, -4.2, -0.7, -0.7

cA210.0, 13.0

cD2-6.0, 12.0

cA016.26

cD0-2.12

Level-1 orScale-1

Level-2 orScale-2

Level-3 orScale-3

- - - -

Wavelet coefficients = {16.2, -2.1, -6.0, 12.0, -1.4, -4.2, -0.7, -0.7}

Dr. M. Jahanur Rahman, R.U. 86

Multi-level Decomposition of a Series with Wavelets

Dr. M. Jahanur Rahman, R.U. 87

DWT Analysis

IDWT Reconstruction

Discrete Wavelet Transform (IDWT).

The input can be reconstructed using the Inverse

Frequency Domain Application-1 Fourier Transformation

Spectral density

Determining periods

Removing Periodicities

Modeling and Forecasting

Dr. M. Jahanur Rahman, R.U. 89

Autocorrelation function or correlogram is used for analyzing time series in time domain.

Time series rk

k

Correlogram

t

Yt

Periodic process with noise

Why Frequency Domain?

Dr. M. Jahanur Rahman, R.U. 90

The steps of analyzing the data

Plot the time series

Plot the correlogram (to suspect the periodicity)

Plot the spectrum (to identify the periodicities)

Ik

wk

rk

k

Yk

t

Dr. M. Jahanur Rahman, R.U. 91

The spectrum shows prominent spikes which represents the periodicities inherent in the data.

The period corresponding the any value of may be computed by

While the correlogram indicate the presence of periodicities in the data, the spectral analysis helps identify the significant periodicities themselves

Periodicities

Dr. M. Jahanur Rahman, R.U. 92

Ik

wk

Ik

wk

Ik

wk

Ik

wk

Reconstruct new series by

removing the significance

periodicity one after another as

Removing Periodicities

Dr. M. Jahanur Rahman, R.U. 93

The periodicities are tested for signoficance by defining the

following statistics (Kashyap and Rao 1976)

Where

Statistical significance of periodicities

Dr. M. Jahanur Rahman, R.U. 94

Removing Periodicities

Significant periodicities are removed from the original time series to get a new series , where

The series containing the previous periodicities

where d is no. of periodicities removed (which are known to be significant)

The spectrum of new series is plotted and the spikes are observed until the series become random.

Dr. M. Jahanur Rahman, R.U. 95

Example A monthly time series over the period 1987-2011 is used. Objective: Modeling and forecasting

Time Series ACF

Periodogram PACF

Dr. M. Jahanur Rahman, R.U. 96

Spectral density shows tree significant periodic patterns with frequencies at

with corresponding periods 30 months 12 months 06 months respectively.

Correlogram & SpectrumCorrelogram indicates the presence

of periodicity.

ACF

Periodogram

Dr. M. Jahanur Rahman, R.U. 97

One-period removed series

Two-period removed series

Three-period removed series

Correlogram & Spectrum

Dr. M. Jahanur Rahman, R.U. 98

Model Selection

Seasonally differenced seriesARIMA(2,0,0) = AR(2)

Periodicities removed seriesARIMA(2,0,0) = AR(2)

PAC

PACAC

AC

Dr. M. Jahanur Rahman, R.U. 99

Trail data: January, 1964 – December, 2007Test data: January, 2008 – December, 2008

Forecasting

Dr. M. Jahanur Rahman, R.U. 100

Comparison of Forecasting PerformanceMean Absolute Error (MAE)Root Mean Squared Error (RMSE)Mean Absolute Percentage Error (MAPE)

Lower-value implies best forecast

Dr. M. Jahanur Rahman, R.U. 101

seasonally differenced series Parameters are significant R-square is high DW-statistic is near 2

Periodicities removed series Parameters are significant R-square is moderate DW-statistic is about 2

EstimationAR(2) model

Dr. M. Jahanur Rahman, R.U. 102

Forecasting

RMSE MAE PR series 1.173 4.739D series 0.852 3.914

Dr. M. Jahanur Rahman, R.U. 103

Decomposition

Discrete Wavelet Transformation

Modeling & Forecasting

Humidity & Maximum Temperature of Rajshahi

Data obtained from BARC

Frequency Domain Application-2

Dr. M. Jahanur Rahman, R.U. 104

Monthly Humidity of Rajshahi(1964-2008)

Dr. M. Jahanur Rahman, R.U. 105

Humidity y(t)

A1

A2

A3

A4

D1

D2

D3

D4

Y = A1 + D1

Y = A2 + D2 +D1

Y = A3 + D3 + D2 +D1

Y = A4 + D4 + D3 + D2 +D1

Decom

positi

on by

wavele

t

Trans

form

ation

Monthly humidity of Rajshahi over 1964-2008

Dr. M. Jahanur Rahman, R.U. 106

DecompositionLevel =3 Wavelet : Doubasis-5

Monthly humidity of Rajshahi over 1964-2008

Dr. M. Jahanur Rahman, R.U. 107

Input series

Wavelet Transformatio

nForecasting

Inverse Wavelet

Transformation

Output series

ForecastingInput series

Wavelet Transformatio

n Processing

Inverse Wavelet

Transformation

Output series

Method-1

Method-2

Forecasting Framework

Dr. M. Jahanur Rahman, R.U. 108

Forecasting Framework

Proposed by: Rocha, T. et.al. (2010)

Dr. M. Jahanur Rahman, R.U. 109

Method-1Step-1: The wavelet transformation of type Daubechies-5 and decomposition level 3 is applied to the series ( result in 4 series denoted by and .

Step-2: Use a specific ARIMA model of each one of the constitutive series to forcast its future values, that is,

Step-3: The inverse wavelet transform is used to reconstruct the forecast series as

Dr. M. Jahanur Rahman, R.U. 110

Method-2Step-1: The wavelet transformation of type Daubechies-5 and decomposition level 3 is applied to the series ( result in 4 series denoted by and .

Step-2: Reconstruct the series by removing the high frequency component

Step-3: Use the appropriate ARIMA model to the reconstructed series to forecast the future values;

Dr. M. Jahanur Rahman, R.U. 111

Trail data: January, 1964 – December, 2007

Test data: January, 2008 – December, 2008

Forecasting of Humidity

Dr. M. Jahanur Rahman, R.U. 112

Comparison of Forecasting PerformanceMean Absolute Error (MAE)Root Mean Squared Error (RMSE)Mean Absolute Percentage Error (MAPE)

Lower-value implies best forecast

Dr. M. Jahanur Rahman, R.U. 113

Method MAE RMSE MAPE

Time domain Method 0.1633 0.2187 18.566

Frequency Domain Method-1

0.1336 0.1591 15.3654

Frequency Domain Method-2

0.0654 0.0799 9.1472

Forecasting performance over Jan-2008 to Dec-2008

Forecasting of HumidityModels

Time Domain: ARIMA(0,1,1)(0,1,1)

Frequency Domain-1: ARIMA(8,2,8)(0,0,1), ARIMA(0,1,2)(0,1,1),

ARIMA(0,0,4)(0,1,0), ARIMA(2,0,2)(0,0,0)

Frequency Domain-2: ARIMA(2,1,1)(0,1,1)

Dr. M. Jahanur Rahman, R.U. 114

Frequency Domain Method-2 Time Domain Method

Method MAE RMSE MAPE

Time domain Method 0.1633 0.2187 18.566

Frequency Domain Method-1

0.16835 0.2296 19.001

Frequency Domain Method-2

0.0654 0.0799 9.1472

Forecasting of Humidity

Dr. M. Jahanur Rahman, R.U. 115

Monthly Maximum Temperature of Rajshahi(1964-2009)

Trail data: Jan, 1964 – Dec, 2008

Test data: Jan, 2009 – Dec, 2009

Dr. M. Jahanur Rahman, R.U. 116

Forecasting of Maximum Temperature

Monthly humidity of Rajshahi over 1964-2008

Method MAE RMSE MAPE

Time domain Method 0.7339 1.0507 2.2315

Frequency Domain Method-1

Frequency Domain Method-2

0.6995 0.8286 2.1402

Models

Time Domain: ARIMA(0,0,1)(0,1,1)

Frequency Domain-1: ARIMA(1,2,8)(0,0,1), ARIMA(4,1,4)(0,1,1),

ARIMA(0,0,6)(0,1,1), ARIMA(1,0,4)(0,1,1)

Frequency Domain-2: ARIMA(0,0,1)(0,1,1)

Dr. M. Jahanur Rahman, R.U. 117

Forecasting of Maximum Temperature

Method MAE RMSE MAPE

Time domain Method 0.7339 1.0507 2.2315

Frequency Domain Method-1

Frequency Domain Method-2

0.6995 0.8286 2.1402

Dr. M. Jahanur Rahman, R.U. 118

In modeling time series both time domain and

frequency domain analysis should be done

together.

For high frequency and noisy data, frequency

domain analysis performs better.

Recommendation

Dr. M. Jahanur Rahman, R.U. 119

Than

ks