Ch 9. FORECASTING - Missouri State...

Ch 9. FORECASTING

Time Series Analysis

Time Series Analysis Ch 9. FORECASTING

In this chapter, we assume the model is known exactly, andconsider the calculation of forecasts and their properties for bothdeterministic trend models and ARIMA models.

9.1 Minimum Mean Square Error Forecasting

Suppose the series is available up to time t, namelyY1,Y2, · · · ,Yt . We forecast the value of Yt+`. We call t theforecast origin, ` the lead time, and denote the forecast as Yt(`).We shall develop methods based on minimizing the mean squareforecasting error. It turns out that

Yt(`) = E (Yt+` | Y1,Y2, · · · ,Yt).


9.2 Deterministic Trends

We assume that the model is Yt = µt + Xt , where µt is adeterministic function, and Xt is a white noise with zero mean andvariance γ0. Then

Yt(`) = E (µt+` + Xt+` | Y1, · · · ,Yt)

= E (µt+` | Y1, · · · ,Yt) + E (Xt+` | Y1, · · · ,Yt) = µt+`.

The forecast error is given by

et(`) = Yt+` − Yt(`) = Xt+`.

Since E (et(`)) = 0, the forecast is unbiased. The forecast errorvariance is

Var (et(`)) = Var (Xt+`) = γ0.


Ex. In Exhibits 3.5 and 3.6, we estimate the temperature series inDubuque, Iowa as a cosine trend

µt = 46.2660 + (−26.7079) cos(2πt) + (−2.1697) sin(2πt)

The start time is January 1964, and the end time is December1975. Here µt is a periodic function of period 1 so we will get thesame forecast temperature on the same month each year. Forexample, the forecast temperature at June 1976 is

µ1976+5/12 = µ5/12 = 68.31◦F


9.3 ARIMA Forecasting

For ARIMA models, the forecasts can be expressed in severaldifferent ways.


9.3.1 AR(1)

The model is Yt − µ = φ(Yt−1 − µ) + et . Replace t by t + `:

Yt+` − µ = φ(Yt+`−1 − µ) + et+`

Take conditional expectation E (· | Y1, · · · ,Yt):

Yt(`)− µ = φ(Yt(`− 1)− µ) + E (et+` | Y1, · · · ,Yt)

Yt(`)− µ = φ(Yt(`− 1)− µ) or Yt(`) = µ+ φ(Yt(`− 1)− µ).

The last equation is the difference equation form of the forecast. Then

Yt(`)− µ = φ(Yt(`− 1)− µ) = φ2(Yt(`− 2)− µ) = · · · = φ`(Yt − µ)

So the lead ` forecast may also be expressed as

Yt(`) = µ+ φ`(Yt − µ).

Since |φ| < 1, we have Yt(`) ≈ µ for large `.

In numerical calculation, apply the difference equation form recursively

will accumulate the round off error. We should use many decimal places

in such calculations.Time Series Analysis Ch 9. FORECASTING

The estimated model is:Yt − 74.3293 = 0.5705(Yt−1− 74.3293) + et , and the last observedvalue of the color property is Yt = 67. We get the lead ` forecast

Yt(`) = 74.3293 + (0.5705)`(67− 74.3293).

We can implement a function to calculate Yt(`) for some `:

Yt(1) = 70.14793, Yt(1) = 71.94383,

Yt(1) = 73.88636, Yt(1) = 74.30253.


Now we calculate the forecast error et(`) for AR(1) model. Recall

Yt+` − µ = et+` + φ(Yt+`−1 − µ)

= et+` + φet+`−1 + φ2(Yt+`−2 − µ) = · · ·= et+` + φet+`−1 + · · ·+ φ`−1et+1 + φ`(Yt − µ),

Yt(`)− µ = φ`(Yt − µ).

Therefore, for AR(1) model,

et(`) = Yt+` − Yt(`) = et+` + φet+`−1 + · · ·+ φ`−1et+1.

In particular, et(1) = et+1.


9.3.2 MA(1)

The invertible model is: Yt = µ+ et − θet−1. Replace t by t + 1and take E (· | Y1, · · · ,Yt):

Yt(1) = µ+ E (et+1 | Y1, · · · ,Yt)− θE (et | Y1, · · · ,Yt) = µ− θet .

For ` > 1,

Yt(`) = µ+ E (et+` | Y1, · · · ,Yt)− θE (et+`−1 | Y1, · · · ,Yt) = µ.


9.3.3 The Random Walk with Drift

The nonstationary model is: Yt = Yt−1 + θ0 + et . Then

E (Yt+` | Y1, · · · ,Yt) = E (Yt+`−1 | Y1, · · · ,Yt)+θ0+E (et+` | Y1, · · · ,Yt)

We getYt(`) = Yt(`− 1) + θ0 = · · · = Yt + θ0`.

Now we consider the forecast error.

Yt+` = Yt+`−1+θ0+et+` = · · · = Yt+θ0`+et+`+et+`−1+· · ·+et+1.

Hence

et(`) = Yt+` − Yt(`) = et+` + et+`−1 + · · ·+ et+1.

So the forecast is unbiased as E(et(`)) = 0, and

Var (et(`)) = σ2e`.

In general, an ARIMA process is nonstationary iff Var (et(`))grows without limit.


9.3.4 ARMA(p,q)

For a stationary invertible ARMA(p,q) model, the differenceequation form is

Yt(`) =

p∑i=1

φi Yt(`− i) + θ0 −q∑

j=1

θjE (et+`−j | Y1, · · · ,Yt)

in which

E (et+j | Y1, · · · ,Yt) =

{0 for j > 0

et+j for j ≤ 0

The Yt(`) is the true forecast when ` > 0, but Yt(`) = Yt+` for` ≤ 0.


Ex. Consider the ARMA(1,2) model:Ys = φYs−1 + θ0 + es − θ1es−1 − θ2es−2. Then

Yt(1) = φYt + θ0 − θ1et − θ2et−1,Yt(2) = φYt(1) + θ0 − θ2et ,Yt(`) = φYt(`− 1) + θ0, ` ≥ 3.

The forecast may be expressed explicitly in terms of µ =θ0

1− φ:

Yt(`)− µ = φ`(Yt − µ)− (φ`−1θ1 + φ`−2θ2)et − φ`−1θ2et−1.


For ARMA(p,q) models, the noise terms et−(q−1), · · · , et−1, etappear directly in the computation of the forecasts for leads` = 1, 2, · · · , q. However, for ` > q, the autoregressive portion ofthe difference equation takes over, and we have

Yt(`) = φ1Yt(`−1)+φ2Yt(`−2)+· · ·+φpYt(`−p)+θ0 for ` > q. (1)

So the nature of the forecasts for long lead times will bedetermined by the autoregressive parameters φ1, · · · , φp.

Recall that θ0 = µ(1− φ1 − φ2 − · · · − φp). (1) may be rewrittenas

Yt(`)−µ = φ1[Yt(`−1)−µ]+φ2[Yt(`−2)−µ]+· · ·+φp[Yt(`−p)−µ] for ` > q.(2)


Now we discuss the forecast error {et(`)} for general (stationary ornonstationary) ARIMA models. Appendix G shows that everyARIMA model has truncated linear process representation:

Yt+` = Ct(`) + It(`) for ` ≥ 1 (3)

where Ct(`) is a function of Yt ,Yt−1, · · · , and

It(`) = et+` + Ψ1et+`−1 + Ψ2et+`−2 + · · ·+ Ψ`−1et+1 for ` ≥ 1.(4)

Take E (· | Y1, · · · ,Yt) on (3):

Yt(`) = E (Ct(`) | Y1, · · · ,Yt) + E (It(`) | Y1, · · · ,Yt) = Ct(`)

Therefore,

et(`) = Yt(`)− Yt+` = It(`) = et+` + Ψ1et+`−1 + · · ·+ Ψ`−1et+1.


It implies that the forecasts are unbiased:

E(et(`)) = 0,

and

Var (et(`)) = σ2e

`−1∑j=0

Ψ2j

for ` ≥ 1.

For stationary ARMA models and large `, the variances increasewith upper bounded:

Var (et(`)) ≈ σ2e

∞∑j=0

Ψ2j

= γ0.

For nonstationary ARIMA models, the forecasts are unbiased;however, the Ψj weights does not decay to zero as j increases, sothat the error variances increase without bounded.


9.3.5 Nonstationary Models

Recall that {Yt} ∼ ARIMA(p, d , q) means that

∇dY t = φ1∇dY t−1+φ2∇dY t−2+· · ·+φp∇dY t−p+et−θ1et−1−θ2et−2−· · ·−θqet−q

where

∇dYt = ∇d−1Yt −∇d−1Yt−1 =d∑

i=0

(d

i

)(−1)iYt−i

= Yt −(d

1

)Yt−1 +

(d

2

)Yt−2 − · · ·+

(d

d − 1

)(−1)d−1Yt−d+1 + (−1)dYt−d .

So an ARIMA(p,d,q) model can be naturally expressed as anonstationary ARMA(p+d,q) model. We can get the forecastsYt(t) and the forecast errors et(`) similarly to those for stationaryARMA(p,q) models.


Ex. The ARIMA(1,1,1) model isYt − Yt−1 = φ(Yt−1 − Yt−2) + θ0 + et − θet−1. It has anARMA(2,1) expression:

Yt = (1 + φ)Yt−1 − φYt−2 + θ0 + et − θet−1.

In the above equation, replace t by t + 1, t + 2, or t + `, and takeE (· | Y1, · · · ,Yt). We get the forecasts:

Yt(1) = (1 + φ)Yt − φYt−1 + θ0 − θet ,Yt(2) = (1 + φ)Yt(1)− φYt + θ0,

Yt(`) = (1 + φ)Yt(`− 1)− φYt(`− 2) + θ0.


For all nonstationary ARIMA models, the forecasts are unbiased,

but the variance of forecast error , Var (et(`)) = σ2e

`−1∑j=0

Ψ2j

,

grows without bound. Some examples:

1 for the random walk with drift, Ψj = 1;

2 for IMA(1,1) model, Ψj = 1− θ for j ≥ 1;

3 for ARI(1,1) model, Ψj = (1− φj+1)/(1− φ) for j ≥ 1.

So with nonstationary series, the distant future is quite uncertain.Section 9.9 gives the Summary of Forecasting with Certain ARIMAModels.


9.4 Prediction Limits

9.4.1 Deterministic Trends

The model is: Yt = µt + Xt . Recall that

Yt(`) = µt+`, et(`) = Xt+`, Var (et(`)) = γ0.

Suppose the stochastic component Xt is normally distributed. Then so isthe forecast error et(`). For a given confidence level 1− α, we could usea standard normal percentile z1−α/2 = F−1(1− α/2) (the inversefunction of the cdf of the standard normal distribution), to claim

P

[−z1−α/2 <

Yt+` − Yt(`)√Var (et(`))

]= 1− α (5)

Thus we may be (1− α)100% confident that the future observation Yt+`

will be contained within the prediction limits

Yt(`)± z1−α/2√

Var (et(`)).


Ex. The monthly average temperature series in Dubuque, Iowa ismodeled by a trend

µt = 46.2660 + (−26.7079) cos(2πt) + (−2.1697) sin(2πt)

with√

Var (et(`)) =√γ0 = 3.7◦F . The predicted temperature of

June 1976 is 68.3◦F . Thus with 95% of confidence interval, theaverage June 1976 temperature is

68.3± 1.96(3.7) = 68.3± 7.252, or [61.05◦F , 75.55◦F ].

In practice, the correct forecast error variance will be slightly largerthan Var (et(`)) as we use the estimated parameters.


9.4.2 ARIMA Models

If the white noise terms {et} are normally distributed, then so isthe forecast error et(`). We know that

Var (et(`)) = σ2e

`−1∑j=0

Ψ2j

.

Both σ2e and Ψ-weights must be estimated from the observed timeseries. For large sample sizes, the estimations will have little effecton the actual prediction limits.


Ex. (TS-ch9.R) The AR(1) model estimation of industry color

property in Exhibit 9.1 estimates that

φ = 0.5705, µ = 74.3293, σ2e = 24.8,

and we have

Yt(`) = 74.3293 + (0.5705)`(67− 74.3293),

Var (et(`)) = σ2e

[1− φ2`

1− φ2

].

Thus the 95% confidence interval for `-step-ahead prediction is

74.3293 + (0.5705)`(67− 74.3293)± 1.96 ·

√24.8

(1− 0.57052`

1− 0.57052

).

We can calculated some intervals for ` = 1, 2, 5, 10, etc.


9.5 Forecasting Illustrations

9.5.1 Deterministic Trends

We use the example of temperature series in Dubuque, Iowa toshow how to plot the series with forecast and confidence band.


Ex. 9.2 (cont)

The model fits quite well with a relatively small error variance, theforecast limits are quite close to the fitted trend forecast.


9.5.2 ARIMA Models

Notice how the forecasts approach the mean exponentially as thelead time increases. Also note how the prediction limits increase inwidth.


9.6 Updating ARIMA Forecasts

The origin forecast with origin time t and lead `+ 1 is Yt(`+ 1).Once the observation Yt+1 at time t + 1 is known, we may updatethe forecast as Yt+1(`). The truncated linear process shows that

Yt+`+1 = Ct(`+ 1) + et+`+1 + Ψ1et+` + · · ·+ Ψ`et+1.

Note that Ct(`+ 1) and et+1 are functions of Yt+1,Yt , · · · . TakeE (· | Y1, · · · ,Yt+1) on both sides.

Yt+1(`) = Ct(`+ 1) + Ψ`et+1 = Yt(`+ 1) + Ψ`[Yt+1 − Yt(1)]

where Yt+1 − Yt(1) is the actual forecast error at time t + 1 onceYt+1 has been observed.


Ex. The AR(1) model for the industrial color property in Exhibit9.1 gives

Y35(1) = 70.14793, Y35(2) = 71.94383.

If we now observe the next value Y36 = 65, then we update theforecast for t = 37 as

Y36(1) = 71.94383 + 0.5705(65− 70.14793) = 69.00694.


9.7 Forecast Weights and Exponentially WeightedMoving Averages

We hope to explicitly determine the forecasts from the observedseries Yt ,Yt−1, · · · ,Y1. In general, an invertible ARIMA(p,d,q)process has an inverted form:

Yt = π1Yt−1 + π2Yt−2 + π3Yt−3 + · · ·+ et .

ThenYt+` = π1Yt+`−1 + π2Yt+`−2 + · · ·+ et+`.

Apply E (· | Y1, · · · ,Yt) on both sides for ` = 1, 2, · · · We get

Yt(1) = π1Yt + π2Yt−1 + π3Yt−2 + · · · ,Yt(2) = π1Yt(1) + π2Yt + π3Yt−1 + · · · ,

...


Now we determine the π-weights. An ARIMA(p,d,q) model maybe written as a nonstationary ARMA(p+d,q) model:

Yt = ϕ1Yt−1+ϕ2Yt−2+· · ·+ϕp+dYt−p−d+et−θ1et−1−· · ·−θqet−q.


Ex. Consider the nonstationary IMA(1,1) model:Yt = Yt−1 + et − θet−1. Use (9.7.2) or substituteet = Yt − π1Yt−1 − π2Yt−2 − · · · for et and et−1. We get

πj = θπj−1 = θj−1π1 = (1− θ)θj−1, j ≥ 1.

Therefore,

Yt(1) = (1− θ)Yt + (1− θ)θYt−1 + (1− θ)θ2Yt−2 + · · ·

Here the π-weights decrease exponentially. Thus Yt(1) is called anexponentially weighted moving average (EWMA). We alsohave

Yt(1) = (1− θ)Yt + θYt−1(1) = Yt−1(1) + (1− θ)[Yt − Yt−1(1)].

They show how to update forecasts from origin t − 1 to origin t.


9.8 Forecasting Transformed Series

9.8.1 Differencing

For nonstationary ARIMA models, we use differences to achievestationarity. Two methods of forecasting may be used:

1 forecasting the original nonstationary series,

2 forecasting the stationary differenced series, then sum theterms to obtain the forecast of original series.

Both methods will lead to the same forecast for any type of lineartransformation with constant coefficients, including differences.Because taking conditional expectation is a linear function.


9.8.2 Log Transformations

Let Yt denote the original series value and let Zt = log(Yt). Wealways have

E (Yt+` | Y1, · · · ,Yt) ≥ exp[E (Zt+` | Z1, · · · ,Zt)]

with equality holding only in trivial cases. Thus exp[Zt(`)] is notthe minimum mean square error forecast of Yt+`.Fact: If X has a normal distribution with mean µ and variance σ2,then

E(exp(X)) = exp

[µ+

σ2

2

].


In our case, µ = E (Zt+` | Z1, · · · ,Zt) and

σ2 = Var (Zt+` | Zt,Zt−1, · · · ,Z1)

= Var(

Zt(`) + et(`) | Zt,Zt−1, · · · ,Z1

)= Var (Ct(`) + et(`) | Zt,Zt−1, · · · ,Z1)

= Var (et(`) | Zt,Zt−1, · · · ,Z1)

= Var (et(`)) .

Thus the minimum mean square error forecast in the original seriesis given by

Yt(`) = exp

{Zt(`) +

1

2Var (et(`))

}.


However, if Zt has a normal distribution, then Yt = exp(Zt) has alognormal distribution, for which a criterion based on the meanabsolute error would be better. The optimal forecast for thiscriterion is the median of Yt+` conditional on Yt ,Yt−1, · · · ,Y1.The log function preserves the mean. So in this case

Yt(`) = mean [Yt+` | Yt , · · · ,Y1]

= exp [mean(Zt+` | Zt · · · ,Z1)]

= exp[Zt(`)

]= expµ.


9.9 Summary of Forecasting with Certain ARIMAModels

This section includes a summary of Yt(`), et(`), Var (et(`)), andΨj , for AR(1), MA(1), IMA(1,1), IMA(2,2) models.


Ch 9. FORECASTING - Missouri State...

Documents

Transcript of Ch 9. FORECASTING - Missouri State...