Lesson 9: Autoregressive-Moving Average (ARMA) models · Lesson 9: Autoregressive-Moving Average...

Lesson 9: Autoregressive-MovingAverage (ARMA) models

Umberto Triacca

Dipartimento di Ingegneria e Scienze dell’Informazione e MatematicaUniversita dell’Aquila,

[email protected]

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Introduction

We have seen that in the class of stationary, zero mean,Gaussian processes the probabilistic structure of a stochasticprocess is completly characterized by the autocovariancefunction.


Autocovariance function

Stationary, zero mean, Gaussian process

&%'$

DGP

?

γx(k)

��7

��

��x1, ..., xT


Introduction

However, in general, to know the autocovariance functionmeans to know a sequence composed by an infinite number ofelements.

We have to estimate a infinite number of parameters

γx(0), γx(1), γx(2), ...,

from observed data.

This mission is impossible


Introduction

We introduce a very important class of stochastic processes,which autocovariance functions depend on a finite number ofunknown parameters:

the class of the AutoregRessive Moving Average (ARMA)processes.


Autoregressive-Moving Average (ARMA) models

Definition. The process {xt ; t ∈ Z} is an autoregressivemoving average process of order (p, q), denoted with

xt ∼ ARMA(p, q),

if

xt −φ1xt−1− ...−φpxt−p = ut + θ1ut−1 + ...+ θqut−q ∀t ∈ Z,

where ut ∼ WN(0, σ2u), and φ1, ..., φp, θ1, ..., θq are p + q

constants and the polynomials

φ(z) = 1− φ1z − ...− φpzp

andθ(z) = 1 + θ1z ... + θqz

q

have no common factors.Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


For q = 0 the process reduces to an autoregressive process oforder p, denoted with xt ∼ AR(p),

xt − φ1xt−1 − ...− φpxt−p = ut ∀t ∈ Z,

For p = 0 to a moving average process of order q, denotedwith xt ∼ MA(q)

xt = ut + θ1ut−1 + ... + θqut−q ∀t ∈ Z,


An example of Autoregressive-Moving Average

(ARMA) process

The process {xt ; t ∈ Z} defined by

xt = 0.3xt−1 + ut + 0.7ut−1 ∀t ∈ Z,

where ut ∼ WN(0, σ2u), is an ARMA(1,1) process.

Hereφ(z) = 1− 0.3z

andθ(z) = 1 + 0.7z .


An example of Autoregressive-Moving Average

(ARMA) process

A realizzation of the ARMA(1,1) processxt = 0.3xt−1 + ut + 0.7ut−1 is presented in the following figure.


An example of Autoregressive (AR) process


xt = 0.7xt−1 − 0.5xt−1 + ut ∀t ∈ Z,

where ut ∼ WN(0, σ2u), is an AR(2) process.

Hereφ(z) = 1− 0.7z + 0.5z2


An example of Autoregressive (AR) process

A realizzation of the AR(2) processxt = 0.7xt−1− 0.5xt−2 + ut is presented in the following figure.


An example of Moving Average (MA) process


xt = ut + 0.7ut−1 ∀t ∈ Z,

where ut ∼ WN(0, σ2u), is an MA(1) process.

Hereθ(z) = 1 + 0.7z


An example of Moving Average (MA) process

A realizzation of the MA(1) process xt = ut + 0.7ut−1 ispresented in the following figure.


An example of over-parameterization

Consider the process {xt ; t ∈ Z} defined by

xt = xt−1 − 0.21xt−2 + ut − 0.7ut−1 ∀t ∈ Z,

where ut ∼ WN(0, σ2u).

This process looks like an ARMA(2,1) process but it is not anARMA(2,1) process.


An example of over-parameterization

Here

φ(z) = 1− z + 0.21z2 = (1− 0.7z)(1− 0.3z)

andθ(z) = 1− 0.7z

We note that both polynomials have a common factor, namely1− 0.7z . Discarding the common factor in each leaves

φ∗(z) = 1− 0.3z

andθ∗(z) = 1.

Thus the process is an AR(1) process, defined byxt = 0.3xt−1 + ut


Causal Autoregressive-Moving Average (ARMA)

models

Definition. An ARMA(p, q) process {xt ; t ∈ Z} is causal(strictly, a causal function of {ut ; t ∈ Z}) if there existsconstants ψ0, ψ1, ... such that

∞∑j=0

|ψj | <∞

and

xt =∞∑j=0

ψjut−j ∀t.



Here, it is important to clarify the meaning of equality

xt =∞∑j=0

ψjut−j ∀t

It means that

limn→∞

E

(xt − n∑j=0

ψjut−j

)2 = 0.

The equality is defined in terms of a limit in the quadraticmean.



The following two theorems provide, respectively, acharacterization of the of causality and stationarity of anARMA(p, q) process.



Theorem. An ARMA(p, q) process {xt ; t ∈ Z} is causal ifand only if

φ(z) = 1− φ1z − ...− φpzp 6= 0 for all |z | ≤ 1.

Theorem. An ARMA(p, q) process {xt ; t ∈ Z} is stationaryif and only if

φ(z) = 1− φ1z − ...− φpzp 6= 0 for all |z | = 1.



The causality and the stationarity of an ARMA process dependentirely on the autoregressive parameters and not on themoving-average ones.



Further, we note that if an ARMA(p, q) process is causal, thenis stationary, but stationarity does not imply causality.



Consider, for example, the following AR(1) process:

xt = 3xt−1 + ut

where ut WN(0, σ2u). We have that

φ(z) = 1− 3z 6= 0 for all |z | = 1.

and hence the process is stationary, but non causal since

φ(z) = 1− 3z = 0 for z = 1/3.



An important result: There is a one-to-one correspondencebetween the parameters of a causal ARMA(p,q) process andthe autocovariance function.



It is important to underline that if we consider the set ofautocorrelation functions there is not a one-to-onecorrespondence between the parameters of a causalARMA(p,q) process and the autocorrelation function.



Consider the following two MA(1) processes.

xt = ut + θut−1

where ut ∼ WN(0, σ2u), with |θ| < 1 and

yt = ut +1

θut−1

where ut ∼ WN(0, σ2u).

Sinceθ

1 + θ2=

1/θ

1 + (1/θ)2 ,

we have that both processes share the same autocorrelationfunction. Thus it cannot be used to distinguish between thetwo parametrizations.



This example shows that an MA(1)-process is not uniquelydetermined by its autocorrelation function. There is anidentification problem with the MA(1) models.

In general, (if all roots of θ(z) = 0 are real) there can be 2q

different MA(q) processes with the same autocorrelationfunction.



Definition. An ARMA(p, q) process {xt ; t ∈ Z} is invertible(strictly, an invertible function of {ut ; t ∈ Z}) if there existsconstants π0, π1, ... such that

∞∑j=0

|πj | <∞

and

ut =∞∑j=0

πjxt−j ∀t.



The following theorem provides a necessary and sufficientcondition for the invertibility.

Theorem. An ARMA(p, q) process {xt ; t ∈ Z} is invertibleif and only if

θ(z) = 1 + θ1z + ... + θqzq 6= 0 for all |z | ≤ 1.



We note that an AR(p) process is always invertible, even if itis non-stationary, while an MA(q) process is always stationary,even if it is non-invertible.



The invertibility can be used in order to ensure theidentifiability of MA processes.



In general, (if all roots of θ(z) = 0 are real) there can be 2q

different MA(q) processes with the same autocorrelationfunction, but only one of these is invertible.


Conclusion

In the class of the mean-zero causal and invertible GaussianARMA processes there is a one-to-one correspondencebetween the family of the finite dimensional distributions ofthe process and the finite parametric representation of process.


Conclusion

In the class of the mean-zero causal and invertibleGaussian ARMA processes the probabilistic properties of theprocess are completely characterized by the finite set ofparameters {

φ1, φ2, ..., φp, θ1, θ2, ..., θq, σ2u

}Now, we have to estimate a finite number (p + q + 1) ofparameters from observed data.

This mission is possible.


Conclusion

Zero-mean causal invertible Gaussian ARMA process

&%'$

DGP

?

{φ1, φ2, ..., φp, θ1, θ2, ..., θq, σ2u}

��7

��

��x1, ..., xT


Lesson 9: Autoregressive-Moving Average (ARMA) models · Lesson 9: Autoregressive-Moving Average...

Documents

Transcript of Lesson 9: Autoregressive-Moving Average (ARMA) models · Lesson 9: Autoregressive-Moving Average...