1

Time Series Analysis

2

What is a time series?A time series is any series of data that varies

over time. For exampleQuarterly GDP of LaosHourly price of stocks and sharesWeekly quantity of beer sold in a pub

Each time-series is one realization of a stochastic (i.e. random) process

3

Stationary Time Series

Has constant Mean and constant Variance across time

Has a theoretical covariance between values of Xt that depends only on the difference apart in time

( )tE X 2 2( ) ( )t tVar X E X

( , ) ( )( )

, .t t h t t hCov X X E X X depend

only on h but not on t

5

6

7

8

9

WHITE NOISE PROCESS

Xt = ut ut ~ IID(0, σ2 )

Stationary time series

White Noise

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

10

Stationary time series

Xt = 0.5*Xt-1 + ut ut ~ IID(0, σ2 )

Stationary without drift

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

110 50 100 150 200 250 300 350 400 450

.25

.5

.75

1

1.25

1.5

1.75

2

2.25

2.5

Many Economic Series are not stationary

Share Prices

0 50 100 150 200 250

.35

.4

.45

.5

.55

.6

.65

.7

Exchange Rate

1960 1965 1970 1975

8.7

8.8

8.9

9

9.1

9.2

9.3

Income

12

Non Stationary Time Series

There is no long-run mean for the time-series

and/or

The variance is time dependent and goes to infinity as time approaches to infinity

13

Spurious RegressionThe results of classical econometric theory are derived under the assumption that variables of interests are stationary.

A Spurious or Nonsensical relationship may result when one Non-stationary time series is regressed on one or more Non-stationary time series.

14

15

An example of Non-stationary time series

RANDOM WALKXt = Xt-1 + ut ut ~ IID(0, σ2 )

Xt = X0 + u1 + u2 +…+ ut

Variance: Var(Xt) = Var(X0) + Var(u1) +…+ Var(ut) = 0 + σ2 +…+ σ2

= t σ2

(variance is not constant through time)

16

Non-stationary time series: Random Walk

Xt = Xt-1 + ut ut ~ IID(0, σ2 )

Random Walk

0

0.5

1

1.5

2

2.5

17

What is a “Unit Root”?

If a Non-Stationary Time Series Xt has to be “differenced” d times to make it stationary, then Xt is said to contain d “Unit Roots”. It is customary to denote Xt ~ I(d) which reads “Xt is integrated of order d”

19

20

Establishment of Stationarity Using Differencing of Integrated Series

If Xt ~ I(1), then Zt = Xt – Xt-1 is Stationary

If Xt ~ I(2), then Zt = Xt – Xt-1 – (Xt – Xt-2 ) is Stationary

21

Random Walks

A random walk is as a unit root processA random walk process is “integrated of order one, [I(1)],” meaning a first difference will be stationary, “I(0)”.

23

24

Removing trend and seasonality in a time-series

Economic time series often have a trendOne possibility to remove a linear trend involves regressing the series on t, which can be modeled as xt = 0 + 1t + et

Then analyze the residuals from the detrended series.

25

Seasonality

Sometimes time-series data exhibits some seasonality. Example: Quarterly data on retail sales will tend to jump up in the 4th quarterAs with trends, the series can be seasonally adjusted by adding a set of seasonal dummies in the above regression.

26

Alternatively,

We can simply remove trend or seasonality by calculating first difference or seasonal difference.

27

A trending series or seasonal series cannot be stationary, since the mean is changing over time.After the adjustment, the remaining maybe stationary and can be analyzed.

28

29

Differencing the series Differencing the series to achieve stationarityto achieve stationarity

Identify model to be Identify model to be tentatively entertainedtentatively entertained

Estimate the parameters Estimate the parameters of the tentative modelof the tentative model

Diagnostic checking. Is Diagnostic checking. Is the model adequate?the model adequate?

NoNo

YesYesUse the model for Use the model for forecastingforecasting

Times-Series Analysis --- (Times-Series Analysis --- (Box-Jenkins Box-Jenkins Approach analysis, ARIMA analysis)Approach analysis, ARIMA analysis)

30

ARIMA AnalysisARIMA Analysis

• We use an autoregressive integrated moving We use an autoregressive integrated moving average (ARIMA) model to fit the data.average (ARIMA) model to fit the data.

• If time-series data is already stationary, can If time-series data is already stationary, can use autoregressive moving average (ARIMA) use autoregressive moving average (ARIMA) model to fit the data.model to fit the data.

We will use ARMA model to fit a stationary time-series data.

ARMA (p, q) refers to a model with p autoregressive terms and q moving average terms.An ARMA (p,q) process is a stationary process in the following form :

1 1 2 2 1 1 2 2... ...

{ } .t t t p t p t t t q t q

t

X X X X

Where is a white noise process

33

An AR(1) Process (ARMA(p=1,q=0))

An autoregressive process of order one [AR(1)] can be characterized as one where xt = xt-1 + et , t = 1, 2,… with et being an iid sequence with mean 0 and variance e

2

For this process to be stationary, it must be the case that || < 1

34

Random Walks

A random walk is an AR(1) model where = 1 A random walk series is not stationary.

36

An MA(1) Process (ARMA(p=0,q=1))

A moving average process of order one [MA(1)] can be characterized as one where xt = et + 1et-1, t = 1, 2, … with et being an iid sequence with mean 0 and variance 2

e

This is a stationary sequence. Variables 1 period apart are correlated, but 2 periods apart they are not.

37

How to choose p and q? Check How to choose p and q? Check ACF and PACF graphsACF and PACF graphs

• Autocorrelation function (ACF)- ratio of sample covariance (at lag k) to sample variance

• Partial autocorrelation function (PACF) – – measures correlation between (time series) observations that are k time periods apart after controlling for correlations at intermediate lags (i.e., lags less than k). In other words, it is the correlation between Yt and Yt-k after removing the effects of intermediate Y’s.

• Correlogram – – graph showing the ACF and the PACF at different lags.

38

Type of Model

Typical Pattern of ACF

Typical Pattern of PACF

AR (p) Decays exponentially or with damped sine wave

pattern or both

Significant spikes through lags p

MA (q) Significant spikes through lags q

Declines exponentially

ARMA (p,q) Exponential decay Exponential decay

Identifying the model by examining Identifying the model by examining patterns of ACF and PACFpatterns of ACF and PACF

39

40

41

42

Diagnostic Checking—Is our model good enough?Diagnostic Checking—Is our model good enough?

How do we know that the model we estimated is a reasonable fit to the data?

Check residuals after the modeling

Rule of thumb: None of the ACF and the PACF are individually statistically significant

Formal test: 2

1

ˆ( 2)

mh

h

Q n nn h

Ljung-Box Q

h is estimated autocorrelation at lag h.

43

44

Some issues in the time-series Some issues in the time-series modelingmodelingDo not know the underlying cause; we still do not understand why the series is behaving this or that way.

Judgmental decisions on the choice of AR, MA, p, q. Also, model identification requires large number of observations but often times-series data is not very long.

The model you fitted in past data may not work well in current or future data.