Advances in automatic time series forecasting

Advances inautomatictime series forecasting

Advances in automatic time series forecasting 1

Rob J Hyndman

Outline

1 Motivation

2 Exponential smoothing

3 ARIMA modelling

4 Time series with complex seasonality

5 Hierarchical and grouped time series

6 Functional time series

7 Grouped functional time series

Advances in automatic time series forecasting Motivation 2

Motivation

1 Common in business to have over 1000products that need forecasting at least monthly.

2 Forecasts are often required by people who areuntrained in time series analysis.

3 Some types of data can be decomposed into alarge number of univariate time series thatneed to be forecast.

Specifications

Automatic forecasting algorithms must:

å determine an appropriate time series model;

å estimate the parameters;

å compute the forecasts with prediction intervals.Advances in automatic time series forecasting Motivation 4

Motivation

Specifications

Motivation

Specifications

Motivation

Specifications

Motivation

Specifications

Example: Cortecosteroid sales

Monthly cortecosteroid drug sales in Australia

1995 2000 2005 2010

Example: Cortecosteroid sales

Automatic ARIMA forecasts

1995 2000 2005 2010

Outline

1 Motivation

3 ARIMA modelling

Advances in automatic time series forecasting Exponential smoothing 6

Exponential smoothing methods

Seasonal ComponentTrend N A M

Component (None) (Additive) (Multiplicative)

N (None) N,N N,A N,M

A (Additive) A,N A,A A,M

Ad (Additive damped) Ad,N Ad,A Ad,M

M (Multiplicative) M,N M,A M,M

Md (Multiplicative damped) Md,N Md,A Md,M

N,N: Simple exponential smoothing

N,N: Simple exponential smoothingA,N: Holt’s linear method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend methodM,N: Exponential trend method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend methodM,N: Exponential trend methodMd,N: Multiplicative damped trend method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend methodM,N: Exponential trend methodMd,N: Multiplicative damped trend methodA,A: Additive Holt-Winters’ method

N,N: Simple exponential smoothingA,N: Holt’s linear methodAd,N: Additive damped trend methodM,N: Exponential trend methodMd,N: Multiplicative damped trend methodA,A: Additive Holt-Winters’ methodA,M: Multiplicative Holt-Winters’ method

There are 15 separate exponential smoothingmethods.

There are 15 separate exponential smoothingmethods.Each can have an additive or multiplicativeerror, giving 30 separate models.

General notation E T S : ExponenTial Smoothing

Examples:A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errorsAdvances in automatic time series forecasting Exponential smoothing 8

General notation E T S : ExponenTial Smoothing

Examples:A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errorsAdvances in automatic time series forecasting Exponential smoothing 8

General notation E T S : ExponenTial Smoothing↑

TrendExamples:

A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errorsAdvances in automatic time series forecasting Exponential smoothing 8

General notation E T S : ExponenTial Smoothing↑

Trend SeasonalExamples:

General notation E T S : ExponenTial Smoothing ↑

Error Trend SeasonalExamples:

Innovations state space models

å All ETS models can be written in innovationsstate space form.

å Additive and multiplicative versions give thesame point forecasts but different predictionintervals.

Automatic forecasting

From Hyndman et al. (IJF, 2002):

Apply each of 30 models that are appropriate tothe data. Optimize parameters and initialvalues using MLE (or some other criterion).

Select best method using AIC:

AIC = −2 log(Likelihood) + 2p

where p = # parameters.

Produce forecasts using best method.

Obtain prediction intervals using underlyingstate space model.

Method performed very well in M3 competition.Advances in automatic time series forecasting Exponential smoothing 9

Exponential smoothing

Automatic ETS forecasts

1995 2000 2005 2010

Exponential smoothing

library(forecast)fit <- ets(h02)fcast <- forecast(fit)plot(fcast)

Automatic ETS forecasts

1995 2000 2005 2010

Exponential smoothing> fitETS(M,Md,M)

Smoothing parameters:alpha = 0.3318beta = 4e-04gamma = 1e-04phi = 0.9695

Initial states:l = 0.4003b = 1.0233s = 0.8575 0.8183 0.7559 0.7627 0.6873 1.2884

1.3456 1.1867 1.1653 1.1033 1.0398 0.9893

sigma: 0.0651

AIC AICc BIC-121.97999 -118.68967 -65.57195

References

Hyndman, Koehler, Snyder, Grose (2002). “Astate space framework for automaticforecasting using exponential smoothingmethods”. International Journal of Forecasting18(3), 439–454

Hyndman, Koehler, Ord, Snyder (2008).Forecasting with exponential smoothing: thestate space approach. Berlin: Springer-Verlag.www.exponentialsmoothing.net

Hyndman (2012). forecast: Forecastingfunctions for time series.cran.r-project.org/package=forecast

Outline

1 Motivation

3 ARIMA modelling

Advances in automatic time series forecasting ARIMA modelling 14

How does auto.arima() work?

A non-seasonal ARIMA process

φ(B)(1− B)dyt = c + θ(B)εt

Need to select appropriate orders p,q,d, andwhether to include c.

A non-seasonal ARIMA process

φ(B)(1− B)dyt = c + θ(B)εt

Need to select appropriate orders p,q,d, andwhether to include c.

Hyndman & Khandakar (JSS, 2008) algorithm:Select no. differences d via KPSS unit root test.Select p,q, c by minimising AIC.Use stepwise search to traverse model space,starting with a simple model and consideringnearby variants.

A seasonal ARIMA process

Φ(Bm)φ(B)(1− B)d(1− Bm)Dyt = c + Θ(Bm)θ(B)εt

Need to select appropriate orders p,q,d, P,Q,D, andwhether to include c.

Hyndman & Khandakar (JSS, 2008) algorithm:Select no. differences d via KPSS unit root test.Select D using OCSB unit root test.Select p,q, P,Q, c by minimising AIC.Use stepwise search to traverse model space,starting with a simple model and consideringnearby variants.

Auto ARIMA

1995 2000 2005 2010

Auto ARIMA

fit <- auto.arima(h02)fcast <- forecast(fit)plot(fcast)

1995 2000 2005 2010

Auto ARIMA

> fitSeries: h02ARIMA(3,1,3)(0,1,1)[12]

Coefficients:ar1 ar2 ar3 ma1 ma2 ma3

-0.3648 -0.0636 0.3568 -0.4850 0.0479 -0.353s.e. 0.2198 0.3293 0.1268 0.2227 0.2755 0.212

sma1-0.5931

s.e. 0.0651

sigma^2 estimated as 0.002706: log likelihood=290.25AIC=-564.5 AICc=-563.71 BIC=-538.48

References

Hyndman, Khandakar (2008). “Automatic timeseries forecasting : the forecast package for R”.Journal of Statistical Software 26(3)

Hyndman (2011). “Major changes to theforecast package”.robjhyndman.com/researchtips/forecast3/.

Hyndman (2012). forecast: Forecastingfunctions for time series.cran.r-project.org/package=forecast

Outline

1 Motivation

3 ARIMA modelling

Advances in automatic time series forecasting Time series with complex seasonality 21

Examples

US finished motor gasoline products

1992 1994 1996 1998 2000 2002 2004

Examples

Number of calls to large American bank (7am−9pm)

5 minute intervals

3 March 17 March 31 March 14 April 28 April 12 May

Examples

Turkish electricity demand

2000 2002 2004 2006 2008

TBATS model

yt = observation at time t

y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +M∑i=1

s(i)t−mi+ dt

`t = `t−1 + φbt−1 + αdt

bt = (1− φ)b + φbt−1 + βdt

p∑i=1

φidt−i +

q∑j=1

θjεt−j + εt

s(i)t =

ki∑j=1

s(i)j,t

s(i)j,t = s(i)j,t−1 cosλ(i)j + s∗(i)j,t−1 sinλ(i)j + γ(i)1 dt

s(i)j,t = −s(i)j,t−1 sinλ(i)j + s∗(i)j,t−1 cosλ(i)j + γ(i)2 dt

TBATS model

y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +M∑i=1

s(i)t−mi+ dt

`t = `t−1 + φbt−1 + αdt

bt = (1− φ)b + φbt−1 + βdt

p∑i=1

φidt−i +

q∑j=1

θjεt−j + εt

s(i)t =

ki∑j=1

s(i)j,t

Box-Cox transformation

TBATS model

y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +M∑i=1

s(i)t−mi+ dt

`t = `t−1 + φbt−1 + αdt

bt = (1− φ)b + φbt−1 + βdt

p∑i=1

φidt−i +

q∑j=1

θjεt−j + εt

s(i)t =

ki∑j=1

s(i)j,t

M seasonal periods

TBATS model

y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +M∑i=1

s(i)t−mi+ dt

`t = `t−1 + φbt−1 + αdt

bt = (1− φ)b + φbt−1 + βdt

p∑i=1

φidt−i +

q∑j=1

θjεt−j + εt

s(i)t =

ki∑j=1

s(i)j,t

M seasonal periods

global and local trend

TBATS model

y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +M∑i=1

s(i)t−mi+ dt

`t = `t−1 + φbt−1 + αdt

bt = (1− φ)b + φbt−1 + βdt

p∑i=1

φidt−i +

q∑j=1

θjεt−j + εt

s(i)t =

ki∑j=1

s(i)j,t

M seasonal periods

ARMA error

TBATS model

y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +M∑i=1

s(i)t−mi+ dt

`t = `t−1 + φbt−1 + αdt

bt = (1− φ)b + φbt−1 + βdt

p∑i=1

φidt−i +

q∑j=1

θjεt−j + εt

s(i)t =

ki∑j=1

s(i)j,t

M seasonal periods

ARMA error

Fourier-like seasonalterms

TBATS model

y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +M∑i=1

s(i)t−mi+ dt

`t = `t−1 + φbt−1 + αdt

bt = (1− φ)b + φbt−1 + βdt

p∑i=1

φidt−i +

q∑j=1

θjεt−j + εt

s(i)t =

ki∑j=1

s(i)j,t

M seasonal periods

ARMA error

Fourier-like seasonalterms

TBATSTrigonometric

Box-Cox

Seasonal

Examples

fit <- tbats(gasoline)fcast <- forecast(fit)plot(fcast)

Forecasts from TBATS(0.999, 2,2, 1, <52.1785714285714,8>)

1995 2000 2005

Examples

fit <- tbats(callcentre)fcast <- forecast(fit)plot(fcast)

Forecasts from TBATS(1, 3,1, 0.987, <169,5>, <845,3>)

5 minute intervals

3 March 17 March 31 March 14 April 28 April 12 May 26 May 9 June

Examples

fit <- tbats(turk)fcast <- forecast(fit)plot(fcast)

Forecasts from TBATS(0, 5,3, 0.997, <7,3>, <354.37,12>, <365.25,4>)

2000 2002 2004 2006 2008 2010

References

Automatic algorithm described inDe Livera, Hyndman, Snyder (2011).“Forecasting time series with complex seasonalpatterns using exponential smoothing”. Journalof the American Statistical Association106(496), 1513–1527.

Slightly improved algorithm implemented inHyndman (2012). forecast: Forecastingfunctions for time series.cran.r-project.org/package=forecast.

More work required!

Outline

1 Motivation

3 ARIMA modelling

Advances in automatic time series forecasting Hierarchical and grouped time series 28

Introduction

AA AB AC

BA BB BC

CA CB CC

Examples

Manufacturing product hierarchies

Pharmaceutical sales

Net labour turnover

Introduction

AA AB AC

BA BB BC

CA CB CC

Examples

Net labour turnover

Introduction

AA AB AC

BA BB BC

CA CB CC

Examples

Net labour turnover

Introduction

AA AB AC

BA BB BC

CA CB CC

Examples

Net labour turnover

Hierarchical/grouped time series

A hierarchical time series is a collection ofseveral time series that are linked together in ahierarchical structure.

A grouped time series is a collection of timeseries that are aggregated in a number ofnon-hierarchical ways.

Example: daily numbers of calls to HP call centresare grouped by product type and location of callcentre.

Forecasts should be “aggregate consistent”,unbiased, minimum variance.

How to compute forecast intervals?Advances in automatic time series forecasting Hierarchical and grouped time series 30

Notation

K : number of levels inhierarchy (excl. Total).

Yt : observed aggregate of allseries at time t.

YX,t : observation on series X attime t.

Y i,t : vector of all series at level iin time t.

Y t = [Yt,Y1,t, . . . ,YK,t]′

Notation

Y t = [Yt,Y1,t, . . . ,YK,t]′

Notation

Y t = [Yt, YA,t, YB,t, YC,t]′ =

1 1 11 0 00 1 00 0 1

YA,tYB,tYC,t

Y t = [Yt,Y1,t, . . . ,YK,t]′

Notation

1 1 11 0 00 1 00 0 1

︸︷︷︸

YA,tYB,tYC,t

Y t = [Yt,Y1,t, . . . ,YK,t]′

Notation

1 1 11 0 00 1 00 0 1

︸︷︷︸

YA,tYB,tYC,t

︸︷︷︸

Y t = [Yt,Y1,t, . . . ,YK,t]′

Notation

1 1 11 0 00 1 00 0 1

︸︷︷︸

YA,tYB,tYC,t

︸︷︷︸

YK,tY t = SYK,t

Y t = [Yt,Y1,t, . . . ,YK,t]′

Hierarchical dataTotal

AX AY AZ

BX BY BZ

CX CY CZ

YtYA,tYB,tYC,tYAX,tYAY,tYAZ,tYBX,tYBY,tYBZ,tYCX,tYCY,tYCZ,t

1 1 1 1 1 1 1 1 11 1 1 0 0 0 0 0 00 0 0 1 1 1 0 0 00 0 0 0 0 0 1 1 11 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 00 0 1 0 0 0 0 0 00 0 0 1 0 0 0 0 00 0 0 0 1 0 0 0 00 0 0 0 0 1 0 0 00 0 0 0 0 0 1 0 00 0 0 0 0 0 0 1 00 0 0 0 0 0 0 0 1

︸︷︷︸

YAX,tYAY,tYAZ,tYBX,tYBY,tYBZ,tYCX,tYCY,tYCZ,t

︸︷︷︸

Hierarchical dataTotal

AX AY AZ

BX BY BZ

CX CY CZ

YtYA,tYB,tYC,tYAX,tYAY,tYAZ,tYBX,tYBY,tYBZ,tYCX,tYCY,tYCZ,t

1 1 1 1 1 1 1 1 11 1 1 0 0 0 0 0 00 0 0 1 1 1 0 0 00 0 0 0 0 0 1 1 11 0 0 0 0 0 0 0 00 1 0 0 0 0 0 0 00 0 1 0 0 0 0 0 00 0 0 1 0 0 0 0 00 0 0 0 1 0 0 0 00 0 0 0 0 1 0 0 00 0 0 0 0 0 1 0 00 0 0 0 0 0 0 1 00 0 0 0 0 0 0 0 1

︸︷︷︸

YAX,tYAY,tYAZ,tYBX,tYBY,tYBZ,tYCX,tYCY,tYCZ,t

︸︷︷︸

Grouped data

YtYA,tYB,tYX,tYY,tYAX,tYAY,tYBX,tYBY,t

1 1 1 11 1 0 00 0 1 11 0 1 00 1 0 11 0 0 00 1 0 00 0 1 00 0 0 1

︸︷︷︸

YAX,tYAY,tYBX,tYBY,t

︸︷︷︸

Grouped data

YtYA,tYB,tYX,tYY,tYAX,tYAY,tYBX,tYBY,t

1 1 1 11 1 0 00 0 1 11 0 1 00 1 0 11 0 0 00 1 0 00 0 1 00 0 0 1

︸︷︷︸

YAX,tYAY,tYBX,tYBY,t

︸︷︷︸

Forecasts

Key idea: forecast reconciliation

å Ignore structural constraints and forecast everyseries of interest independently.

å Adjust forecasts to impose constraints.

Let Yn(h) be vector of initial forecasts for horizon h,made at time n, stacked in same order as Y t.

Y t = SYK,t . So Yn(h) = Sβn(h) + εh .

βn(h) = E[YK,n+h | Y1, . . . ,Yn].εh has zero mean and covariance matrix Σh.Estimate βn(h) using GLS?

Revised forecasts: Yn(h) = Sβn(h)Advances in automatic time series forecasting Hierarchical and grouped time series 34

Forecasts

Optimal combination forecastsYn(h) = Sβn(h) = S(S′Σ†hS)−1S′Σ†hYn(h)

Σ†h is generalized inverse of Σh.

Problem: Don’t know Σh and hard to estimate.

Solution: Assume εh ≈ SεK,h where εK,h is theforecast error at bottom level.Then Σh ≈ SΩhS′ where Ωh = Var(εK,h).If Moore-Penrose generalized inverse used,then

(S′Σ†S)−1S′Σ† = (S′S)−1S′.

Yn(h) = S(S′S)−1S′Yn(h)

Base forecasts

(S′Σ†S)−1S′Σ† = (S′S)−1S′.

Revised forecasts Base forecasts

(S′Σ†S)−1S′Σ† = (S′S)−1S′.

Optimal combination forecasts

GLS =OLS.

Optimal weighted average of base forecasts.

Computational difficulties in big hierarchies dueto size of S matrix.

Optimal weights are S(S′S)−1S′

Weights are independent of the data!

GLS =OLS.

Weights: S(S′S)−1S′ =0.75 0.25 0.25 0.250.25 0.75 −0.25 −0.250.25 −0.25 0.75 −0.250.25 −0.25 −0.25 0.75

AA AB AC

BA BB BC

CA CB CC

Weights: S(S′S)−1S′ =

0.69 0.23 0.23 0.23 0.08 0.08 0.08 0.08 0.08 0.08 0.08 0.08 0.080.23 0.58 −0.17 −0.17 0.19 0.19 0.19 −0.06 −0.06 −0.06 −0.06 −0.06 −0.060.23 −0.17 0.58 −0.17 −0.06 −0.06 −0.06 0.19 0.19 0.19 −0.06 −0.06 −0.060.23 −0.17 −0.17 0.58 −0.06 −0.06 −0.06 −0.06 −0.06 −0.06 0.19 0.19 0.190.08 0.19 −0.06 −0.06 0.73 −0.27 −0.27 −0.02 −0.02 −0.02 −0.02 −0.02 −0.020.08 0.19 −0.06 −0.06 −0.27 0.73 −0.27 −0.02 −0.02 −0.02 −0.02 −0.02 −0.020.08 0.19 −0.06 −0.06 −0.27 −0.27 0.73 −0.02 −0.02 −0.02 −0.02 −0.02 −0.020.08 −0.06 0.19 −0.06 −0.02 −0.02 −0.02 0.73 −0.27 −0.27 −0.02 −0.02 −0.020.08 −0.06 0.19 −0.06 −0.02 −0.02 −0.02 −0.27 0.73 −0.27 −0.02 −0.02 −0.020.08 −0.06 0.19 −0.06 −0.02 −0.02 −0.02 −0.27 −0.27 0.73 −0.02 −0.02 −0.020.08 −0.06 −0.06 0.19 −0.02 −0.02 −0.02 −0.02 −0.02 −0.02 0.73 −0.27 −0.270.08 −0.06 −0.06 0.19 −0.02 −0.02 −0.02 −0.02 −0.02 −0.02 −0.27 0.73 −0.270.08 −0.06 −0.06 0.19 −0.02 −0.02 −0.02 −0.02 −0.02 −0.02 −0.27 −0.27 0.73

AA AB AC

BA BB BC

CA CB CC

Weights: S(S′S)−1S′ =

0.69 0.23 0.23 0.23 0.08 0.08 0.08 0.08 0.08 0.08 0.08 0.08 0.080.23 0.58 −0.17 −0.17 0.19 0.19 0.19 −0.06 −0.06 −0.06 −0.06 −0.06 −0.060.23 −0.17 0.58 −0.17 −0.06 −0.06 −0.06 0.19 0.19 0.19 −0.06 −0.06 −0.060.23 −0.17 −0.17 0.58 −0.06 −0.06 −0.06 −0.06 −0.06 −0.06 0.19 0.19 0.190.08 0.19 −0.06 −0.06 0.73 −0.27 −0.27 −0.02 −0.02 −0.02 −0.02 −0.02 −0.020.08 0.19 −0.06 −0.06 −0.27 0.73 −0.27 −0.02 −0.02 −0.02 −0.02 −0.02 −0.020.08 0.19 −0.06 −0.06 −0.27 −0.27 0.73 −0.02 −0.02 −0.02 −0.02 −0.02 −0.020.08 −0.06 0.19 −0.06 −0.02 −0.02 −0.02 0.73 −0.27 −0.27 −0.02 −0.02 −0.020.08 −0.06 0.19 −0.06 −0.02 −0.02 −0.02 −0.27 0.73 −0.27 −0.02 −0.02 −0.020.08 −0.06 0.19 −0.06 −0.02 −0.02 −0.02 −0.27 −0.27 0.73 −0.02 −0.02 −0.020.08 −0.06 −0.06 0.19 −0.02 −0.02 −0.02 −0.02 −0.02 −0.02 0.73 −0.27 −0.270.08 −0.06 −0.06 0.19 −0.02 −0.02 −0.02 −0.02 −0.02 −0.02 −0.27 0.73 −0.270.08 −0.06 −0.06 0.19 −0.02 −0.02 −0.02 −0.02 −0.02 −0.02 −0.27 −0.27 0.73

Features and problemsYn(h) = S(S′S)−1S′Yn(h)

Var[Yn(h)] = SΩhS′ = Σh.

Covariates can be included in base forecasts.Point forecasts are always consistent.Need to estimate Ωh to produce prediction intervals.Very simple and flexible method. Can work with anyhierarchical or grouped time series.

Hyndman, Ahmed, Athanasopoulos, Shang (2011).“Optimal combination forecasts for hierarchicaltime series”. Computational Statistics and DataAnalysis 55(9), 2579–2589Hyndman, Ahmed, Shang (2011). hts: Hierarchicaltime series. cran.r-project.org/package=hts

Outline

1 Motivation

3 ARIMA modelling

Advances in automatic time series forecasting Functional time series 39

Fertility rates

Some notationLet yt,x be the observed data in period t at age x,t = 1, . . . ,n.

yt,x = ft(x) + σt(x)εt,x

ft(x) = µ(x) +K∑

βt,k φk(x) + et(x)

Estimate ft(x) using penalized regression splines.

Estimate µ(x) as mean ft(x) across years.

Estimate βt,k and φk(x) using functional (weighted)principal components.

εt,xiid∼ N(0,1) and et(x)

iid∼ N(0, v(x)).

ft(x) = µ(x) +K∑

iid∼ N(0, v(x)).

ft(x) = µ(x) +K∑

iid∼ N(0, v(x)).

ft(x) = µ(x) +K∑

iid∼ N(0, v(x)).

ft(x) = µ(x) +K∑

iid∼ N(0, v(x)).

Fertility application

15 20 25 30 35 40 45 50

Australia fertility rates (1921−2006)

tility

Fertility model

15 20 25 30 35 40 45 50

1920 1960 2000

15 20 25 30 35 40 45 50

1920 1960 2000

Functional time series model

ft(x) = µ(x) +K∑

The eigenfunctions φk(x) show the mainregions of variation.The scores βt,k are uncorrelated byconstruction. So we can forecast each βt,k usinga univariate time series model.Univariate ARIMA models used for automaticforecasting of scores.

ft(x) = µ(x) +K∑

Forecasts

ft(x) = µ(x) +K∑

Forecasts

ft(x) = µ(x) +K∑

where vn+h,k = Var(βn+h,k | β1,k, . . . , βn,k)and y = [y1,x, . . . , yn,x].

E[yn+h,x | y] = µ(x) +K∑

βn+h,k φk(x)

Var[yn+h,x | y] = σ2µ(x) +

K∑k=1

vn+h,k φ2k(x) + σ2

t (x) + v(x)

Forecasts of ft(x)

15 20 25 30 35 40 45 50

tility

Forecasts of ft(x)

15 20 25 30 35 40 45 50

tility

Forecasts of ft(x)

15 20 25 30 35 40 45 50

tility

Forecasts of ft(x)

15 20 25 30 35 40 45 50

tility

80% prediction intervals

References

Hyndman, Ullah (2007). “Robust forecasting ofmortality and fertility rates: A functional dataapproach”. Computational Statistics and DataAnalysis 51(10), 4942–4956

Hyndman, Shang (2009). “Forecastingfunctional time series (with discussion)”.Journal of the Korean Statistical Society 38(3),199–221

Hyndman (2012). demography: Forecastingmortality, fertility, migration and populationdata.cran.r-project.org/package=demography

Outline

1 Motivation

3 ARIMA modelling

Advances in automatic time series forecasting Grouped functional time series 48

Forecasting groups

Let ft,j(x) be the smoothed mortality rate for age xin group j in year t.

Groups may be males and females.

Groups may be states within a country.

Expected that groups will behave similarly.

Coherent forecasts do not diverge over time.

Existing functional models do not imposecoherence.

Forecasting groups

Forecasting the coefficients

ft(x) = µ(x) +K∑

We use ARIMA models for each coefficientβ1,j,k, . . . , βn,j,k.The ARIMA models are non-stationary for thefirst few coefficients (k = 1,2)

Non-stationary ARIMA forecasts will diverge.Hence the mortality forecasts are not coherent.

ft(x) = µ(x) +K∑

Male fts model

0 20 40 60 80

µ M(x

0 20 40 60 80

φ 1(x

1960 2000

0 20 40 60 80

φ 2(x

1960 2000

0 20 40 60 80

φ 3(x

1960 2000

Australian male death rates

Female fts model

0 20 40 60 80

µ F(x

0 20 40 60 80

φ 1(x

1960 2000

0 20 40 60 80

φ 2(x

1960 2000

0 20 40 60 80

φ 3(x

1960 2000

Australian female death rates

Australian mortality forecasts

0 20 40 60 80 100

(a) Males

0 20 40 60 80 100

(b) Females

Mortality product and ratios

Key idea

Model the geometric mean and the mortality ratioinstead of the individual rates for each sexseparately.

pt(x) =√ft,M(x)ft,F(x) and rt(x) =

√ft,M(x)

/ft,F(x).

Product and ratio are approximatelyindependent

Ratio should be stationary (for coherence) butproduct can be non-stationary.

Key idea

√ft,M(x)

/ft,F(x).

Key idea

√ft,M(x)

/ft,F(x).

Key idea

√ft,M(x)

/ft,F(x).

Mortality rates

Model product and ratios

√ft,M(x)

/ft,F(x).

log[pt(x)] = µp(x) +K∑

βt,kφk(x) + et(x)

log[rt(x)] = µr(x) +L∑`=1

γt,`ψ`(x) + wt(x).

γt,` restricted to be stationary processes:either ARMA(p,q) or ARFIMA(p,d,q).No restrictions for βt,1, . . . , βt,K.Forecasts: fn+h|n,M(x) = pn+h|n(x)rn+h|n(x)

fn+h|n,F(x) = pn+h|n(x)/rn+h|n(x).

√ft,M(x)

/ft,F(x).

βt,kφk(x) + et(x)

γt,`ψ`(x) + wt(x).

√ft,M(x)

/ft,F(x).

βt,kφk(x) + et(x)

γt,`ψ`(x) + wt(x).

√ft,M(x)

/ft,F(x).

βt,kφk(x) + et(x)

γt,`ψ`(x) + wt(x).

Product model

0 20 40 60 80

µ P(x

0 20 40 60 80

φ 1(x

1960 2000

0 20 40 60 80

φ 2(x

1960 2000

0 20 40 60 80

φ 3(x

1960 2000

Ratio model

0 20 40 60 80

µ R(x

0 20 40 60 80

φ 1(x

1960 2000

0 20 40 60 80

φ 2(x

1960 2000

0 20 40 60 80

φ 3(x

1960 2000

Product forecasts

0 20 40 60 80 100

Ratio forecasts

0 20 40 60 80 100

Coherent forecasts

0 20 40 60 80 100

(a) Males

0 20 40 60 80 100

(b) Females

Ratio forecasts

0 20 40 60 80 100

Independent forecasts

0 20 40 60 80 100

Coherent forecasts

Life expectancy forecasts

1960 1980 2000 2020

8085Life expectancy difference: F−M

1960 1980 2000 2020

Coherent forecasts for J groups

pt(x) = [ft,1(x)ft,2(x) · · · ft,J(x)]1/J

and rt,j(x) = ft,j(x)/pt(x),

βt,kφk(x) + et(x)

log[rt,j(x)] = µr,j(x) +L∑

γt,l,jψl,j(x) + wt,j(x).

pt(x) and all rt,j(x)are approximatelyindependent.

Ratios satisfy constraintrt,1(x)rt,2(x) · · · rt,J(x) = 1.

log[ft,j(x)] = log[pt(x)rt,j(x)]

pt(x) = [ft,1(x)ft,2(x) · · · ft,J(x)]1/J

βt,kφk(x) + et(x)

pt(x) = [ft,1(x)ft,2(x) · · · ft,J(x)]1/J

βt,kφk(x) + et(x)

pt(x) = [ft,1(x)ft,2(x) · · · ft,J(x)]1/J

βt,kφk(x) + et(x)

pt(x) = [ft,1(x)ft,2(x) · · · ft,J(x)]1/J

βt,kφk(x) + et(x)

µj(x) = µp(x) + µr,j(x) is group mean

zt,j(x) = et(x) + wt,j(x) is error term.

γt,` restricted to be stationary processes:either ARMA(p,q) or ARFIMA(p,d,q).

No restrictions for βt,1, . . . , βt,K.

log[ft,j(x)] = log[pt(x)rt,j(x)] = log[pt(x)] + log[rt,j]

= µj(x) +K∑

βt,kφk(x) +L∑`=1

γt,`,jψ`,j(x) + zt,j(x)

= µj(x) +K∑

References

Hyndman, Booth, Yasmeen (2012). “Coherentmortality forecasting: the product-ratio methodwith functional time series models”.Demography, to appear

Hyndman (2012). demography: Forecastingmortality, fertility, migration and populationdata.cran.r-project.org/package=demography

For further information

robjhyndman.com

Slides and references for this talk.

Links to all papers and books.

Links to R packages.

A blog about forecasting research.

Advances in automatic time series forecasting

Technology

Transcript of Advances in automatic time series forecasting