Stationary Time Series Slides

7/30/2019 Stationary Time Series Slides

1/17

Covariance Stationary Time Series

Stochastic Process: sequence of rvs ordered by time

{Yt}

= {. . . , Y

1, Y0, Y1, . . .}

Defn: {Yt} is covariance stationary if

E[Yt] = for all t

cov(Yt, Ytj) = E[(Yt )(Ytj )] = j forall t and any j

Remarks

j = jth lag autocovariance; 0 = var(Yt)

j = j/0 = jth lag autocorrelation

Example: Independent White Noise (IW N(0, 2))

Yt = t, t iid (0, 2)E[Yt] = 0, var(Yt) =

2, j = 0, j 6= 0

Example: Gaussian White Noise (GW N(0, 2))

Yt = t, t iid N(0, 2)

Example: White Noise (W N(0, 2))

Yt = t

E[t] = 0, var(t) = 2, cov(t, tj) = 0


2/17

Nonstationary Processes

Example: Deterministically trending process

Yt = 0 + 1t + t, t

W N(0, 2)

E[Yt] = 0 + 1t depends on t

Note: A simple detrending transformation yield a station-

ary process:

Xt = Yt 0 1t = t

Example: Random Walk

Yt = Yt1 + t, t W N(0, 2), Y0 is fixed

= Y0 +tX

j=1

j var(Yt) = 2t depends on t

Note: A simple detrending transformation yield a station-

ary process:

Yt = Yt Yt1 = t

Wolds Decomposition Theorem

Any covariance stationary time series {Yt} can be repre-

sented in the form

Yt = + X

j=0

jtj, t W N(0, 2)

0 = 1,X

j=0

2j <

Properties:

E[Yt] =

0 = var(Yt) = 2X

j=0

2j <

j = E[(Yt )(Ytj )]= E[(t + 1t1 + + jtj + )

(t

j + 1t

j

1 + )

= 2(j + j+11 + )

= 2X

k=0

kk+j


3/17

Autoregressive moving average models (ARMA) Models

(Box-Jenkins 1976)

Idea: Approximate Wold form of stationary time series

by parsimonious parametric models (stochastic difference

equations)

ARMA(p,q) model:

Yt = 1(Yt1 ) + + p(Yt2 )+t + 1t1 + + qtq

t W N(0, 2)Lag operator notation:

(L)(Yt ) = (L)t(L) = 1 1L pLp(L) = 1 + 1L + + qL

q

Stochastic difference equation

(L)Xt = wt

Xt = Yt , wt = (L)t

ARMA(1,0) Model (1st order SDE)

Yt = Yt1 + t, t W N(0, 2)Solution by recursive substitution:

Yt = t+1Y1 + tY0 + t0 + + t1 + t

= t+1Y1 +tX

i=0

iti

= t+1Y1 +tX

i=0

iti, i = i

Alternatively, solving forward j periods from time t :

Yt+j = j+1Yt1 + jt + + t+j1 + t+j

= j+1Yt1 +jX

i=0

it+ji

Dynamic Multiplier:

dYjd0

= dYt+jdt

= j = j


4/17

Impulse Response Function (IRF)

Plot j vs. j

Cumulative impact (up to horizon j)

jXi=1

j

Long-run cumulative impact

Xi=1

j = (1)

= (L) evaluated at L = 1

Stability and Stationarity Conditions

If || < 1 then

limj

j = limj

j = 0

and the stationary solution (Wold form) for the AR(1)

becomes.

Yt =X

j=0

jtj =X

j=0

jtj

This is a stable (non-explosive) solution. Note that

(1) =X

j=0

j =1

1 <

If = 1 then

Yt = Y0 +

tXj=0

j, j = 1, (1) =

which is not stationary or stable.


5/17

AR(1) in Lag Operator Notation

(1 L)Yt = tIf || < 1 then

(1 L)1 = Xj=0

jLj = 1 + L + 2L2 +

such that

(1 L)1(1 L) = 1Trick to find Wold form:

Yt = (1 L)1(1 L)Yt = (1 L)1t=

Xj=0

jLjt

=X

j=0

jtj

= X

j=0jtj, j = j

Moments of Stationary AR(1)

Mean adjusted form:

Yt

= (Yt

1

) + t, t

W N(0, 2), || < 1

Regression form:

Yt = c + Yt1 + t, c = (1 )Trick for calculating moments: use stationarity properties

E[Yt] = E[Ytj] for all j

cov(Yt, Ytj) = cov(Ytk, Ytkj) for all k, j

Mean of AR(1)

E[Yt] = c + E[Yt1] + E[t]= c + E[Yt]

E[Yt] =c

1 =


6/17

Variance of AR(1)

0 = var(Yt) = E[(Yt )2] = E[((Yt1 ) + t)2]= 2E[(Yt1 )2] + 2E[(Yt1 )t] + E[2t ]= 20 +

2 (by stationarity)

0 =2

1 2Note: From the Wold representation

0 = var

X

j=0

jtj

= 2

Xj=0

2j =2

1 2

Autocovariances and Autocorrelations

Trick: multiply Yt by Ytj and take expectations

j = E[(Yt

)(Yt

j

)]

= E[(Yt1 )(Ytj )] + E[t(Ytj )]= j1 (by stationarity)

j = j0 = j2

1 2Autocorrelations:

j = j0=

j

00

= j = j

Note: for the AR(1), j = j. However, this is not true

for general ARMA processes.

Autocorrelation Function (ACF)

plot j vs. j


7/17

half-life

0.99 68.970.9 6.580.75 2.410.5 1.000.25 0.50

Table 1: Half lives for AR(1)

Half-Life of AR(1): lag at which IRF decreases by one

half

j = j = 0.5

j ln = ln(0.5) j = ln(0.5)

ln

The half-life is a measure of the speed of mean reversion.

Application: Half-Life of Real Exchange Rates

The real exchange rate is defined as

zt = st pt + ptst = log nominal exchange rate

pt = log of domestic price level

pt = log of foreign price level

Purchasing power parity (PPP) suggests that zt should

be stationary.


8/17

ARMA(p, 0) Model

Mean-adjusted form:

Yt = 1(Yt

1 ) + + p(Ytp ) + tt W N(0, 2)

E[Yt] =

Lag operator notation:

(L)(Yt ) = t(L) = 1

1L

pL

p

Unobserved Components representation

Yt = + Xt

(L)Xt = t

Regression Model formulation

Yt = c + 1Yt1 + + pYtp + t(L)Yt = c + t, c = (1)

Stability and Stationarity Conditions

Trick: Write pth order SDE as a 1st order vector SDE

Xt

Xt1Xt2

...Xtp+1

=

1 2 p

1 0 00 1 . . . 0... ... . . . ...0 0 1 0

Xt1

Xt2Xt3

...Xtp

+

t

00...0

or

t(p1)

= F(pp)

t1(p1)

+ vt(p1)

Use insights from AR(1) to study behavior of VAR(1):

t+j = Fj+1t1 + F

jvt + + Fvt+j1 + vt

Fj = F F F (j times)

Intuition: Stability and stationarity requires

limjFj = 0

Initial value has no impact on eventual level of series.


9/17

Example: AR(2)

Xt = 1Xt1 + 2Xt2 + t

or

" XtXt1

#="

1

21 0# " Xt1Xt2 # + "

t0#

Iterating j periods out gives"Xt+j

Xt+j1

#=

"1 21 0

#j+1 "Xt1Xt2

#

+ " 1 21 0

#j

" t0# + + " 1 2

1 0# " t+j10 # + "

t+j

0

First row gives Xt+j

Xt+j = [f(j+1)11 Xt1 + f

(j+112 Xt2] + f

(j)11 t

+ + f11t+j1 + t+jf

(j)11 = (1, 1) element ofF

j

Note:

F2 =

"1 21 0

# "1 21 0

#=

"21 + 2 12

1 2

#

Result: The ARMA(p, 0) model is covariance stationary

and has Wold representation

Yt = +X

j=0

jtj, 0 = 1

with j = (1, 1) element ofFj provided all of the eigen-

values ofF have modulus less than 1.

Finding Eigenvalues

is an eigenvalue ofF and x is an eigenvector if

Fx = x (F Ip)x = 0 F Ip is singular det(F Ip) = 0

Example: AR(2)

det(F I2) = det1 21 0 ! "

00 #!

= det

1 2

1

!

= 2 1 2


10/17

The eigenvalues ofF solve the reverse characteristic

equation

2 1 2 = 0Using the quadratic equation, the roots satisfy

i =1

q21 + 42

2, i = 1, 2

These root may be real or complex. Complex roots induce

periodic behavior in yt. Recall, if i is complex then

i = a + bi

a = R cos(), b = R sin()

R =q

a2 + b2 = modulus

To see why |i| < 1 implies limjFj = 0 considerthe AR(2) with real-valued eigenvalues. By the spectral

decomposition theorem

F = TT1, T1 = T0

="

1 00 2

#, T =

"t11 t12t21 t22

#,

T1 =

"t11 t12

t21 t22

#

Then

Fj = (TT

1) (TT

1)

= TjT1

and

limj

Fj= T lim

j

jT1 = 0

provided |1| < 1 and |2| < 1.


11/17

Note:

Fj = TjT1

=

"t11 t12t21 t22

# "

j1 0

0 j2

# "t11 t12

t21 t22

#

so that

f(j)11 = (t11t

11)j1 + (t12t

22)j2

= c1j1 + c2

j2 = j

where

c1 + c2 = 1

Then,

limj

j = limj

(c1j1 + c2

j2) = 0

Examples of AR(2) Processes

Yt = 0.6Yt1 + 0.2Yt2 + t1 + 2 = 0.8 < 1

F = " 0.6 0.21 0

#The eigenvalues are found using

i =

q21 + 42

2

1 =0.6 +

q(0.6)2 + 4(0.2)

2

= 0.84

2 =0.6

q(0.6)2 + 4(0.2)

2= 0.24

j = c1(0.84)j + c2(0.24)j


12/17

Yt = 0.5Yt1 0.8Yt2 + t1 + 2 = 0.3 < 1

F = " 0.5 0.81 0

#Note:

21 + 42 = (0.5)2 4(0.8) = 2.95

complex eigenvaluesThen

i = a bi, i =1

a =12

, b =

q(21 + 42)

2

Here

a =0.5

2= 0.25, b =

2.95

2= 0.86

i = 0.25 0.86i

modulus = R = qa2 + b2 = q(0.25)2 + (0.86)2 = 0.895Polar co-ordinate representation:

i = a + bi s.t. a = R cos(), b = R sin()

= R cos() + R sin()i = R ei

Frequency satisfies

cos() = aR = cos1

aR

period =2


13/17

Here,

R = 0.895

= cos1

0.25

0.985

= 1.29

period =

2

1.29 = 4.9Note: the period is the length of time required for the

process to repeat a full cycle.

Note: The IRF has the form

j = c1

j

1 + c2

j

2 Rj[cos(j) + sin(j)]

Stationarity Conditions on Lag Polynomial (L)

Consider the AR(2) model in lag operator notation

(1 1L 2L2)Xt = (L)Xt = tFor any variable z, consider the characteristic equation

(z) = 1 1z 2z2 = 0By the fundamental theorem of algebra

1 1z 2z2 = (1 1z)(1 2z)

so that

z1 =1

1, z2 =

1

2

are the roots of the characteristic equation. The values

1 and 2 are the eigenvalues ofF.

Note: If1+2 = 1 then (z = 1) = 1(1+2) = 0and z = 1 is a root of (z) = 0.


14/17

Result: The inverses of the roots of the characteristic

equation

(z) = 1 1z 2z2 pzp = 0are the eigenvalues of the companion matrix F. There-

fore, the AR(p) model is stable and stationary provided

the roots of (z) = 0 have modulus greater than unity

(roots lie outside the complex unit circle).

Remarks:

1. The reverse characteristic equation for the AR(p) is

zp 1zp1 2zp2 p1z p = 0This is the same polynomial equation used to find the

eigenvalues ofF.

2. If the AR(p) is stationary, then

(L) = 1 1L pLp

= (1 1L)(1 2L) (1 pL)where |i| < 1. Suppose, all i are all real. Then

(1 iL)1 =X

j=0

ji L

j

(L)1 = (1 1L)1 (1 pL)1

=

Xj=0

j1L

j

Xj=0

j2L

j

. . .

Xj=0

j2L

j

The Wold solution for Xt may be found using

Xt = (L)1t

=

X

j=0

j1L

j

X

j=0

j2L

j

. . .

X

j=0

j2L

j

t


15/17

3. A simple algorithm exists to determine the Wold form.

To illustrate, consider the AR(2) model. By definition

(L)1 = (1 1L 2L2) = (L),

(L) =

Xj=0

jLj

1 = (L)(L) 1 = (1 1L 2L2)

(1 + 1L + 2L2 + )

Collecting coefficients of powers of L gives

1 = 1 + (1 + 1)L + (2 11 2)L2 + Since all coefficients on powers of L must be equal to

zero, it follows that

1 = 1

2 = 11 + 2

3 = 12 + 21...

j = 1j1 + 2j2

Moments of Stationary AR(p) Model

Yt = 1(Yt1 ) + + p(Ytp ) + tt W N(0, 2)

or

Yt = c + 1Yt1 + + pYtp + tc = (1 ) = 1 + 2 + + p

Note: if = 1 then (1) = 1 = 0 and z = 1 isa root of (z) = 0. In this case we say that the AR(p)

process has a unit root and the process is nonstationary.


16/17

Straightforward algebra shows that

E[Yt] =

0 = 11 + 22 + + pp + 2

j = 1j

1 + 2j

2 + + pj

p

j = 1j1 + 2j2 + + pjp

The recursive equations for j are called the Yule-Walker

equations.

Result: (0, 1, . . . , p1) is determined from the firstp elements of the first column of the (p2 p2) matrix

2[Ip2 (F F)]1

where F is the state space companion matrix for the

AR(p) model.

ARMA(0,1) Process (MA(1) Process)

Yt = + t + t1 = + (L)t(L) = 1 + L, t W N(0, 2)

Moments:

E[Yt] =

var(Yt) = 0 = E[(Yt )2]= E[(t + t1)2]= 2(1 + 2)

1 = E[(Yt

)(Yt

1

)]

= E[(t + t1)(t + t1)]= 2

1 =10

=

1 + 2

j = 0, j > 1


17/17

Remark: There is an identification problem for

0.5 < 1 < 0.5The values and 1 produce the same value of 1. Forexample, = 0.5 and 1 = 2 both produce 1 = 0.4.

Invertibility Condition: The MA(1) is invertible if || < 1

Result: Invertible MA models have infinite order AR rep-

resentations

(Yt ) = (1 + L)t, || < 1= (1 L)t, = (1 L)1(Yt ) = t

Xj=0

()jLj(Yt ) = t

so that

t = (Yt) + (Yt1) + ()2 (Yt2) +

Stationary Time Series Slides

Documents

Transcript of Stationary Time Series Slides