Automatic Forecasting Algorithms - Rob J. Hyndman · 2017-11-18 · Motivation Common in business...

Automatic Forecasting Algorithms 1

2017 Beijing Workshop onForecasting

AutomaticForecastingAlgorithmsRob J Hyndman

robjhyndman.com/beijing2017

Outline

1 Motivation

2 ETS

3 ARIMA models

4 STLM

5 TBATS

6 FASSTER

7 Comparisons

Automatic Forecasting Algorithms Motivation 2

Motivation


Motivation

Common in business to have over 1000 products thatneed forecasting at least monthly.Forecasts are often required by people who areuntrained in time series analysis.

SpecificationsAutomatic forecasting algorithms must:

å determine an appropriate time series model;å estimate the parameters;å compute the forecasts with prediction intervals.


Outline

1 Motivation

2 ETS

3 ARIMA models

4 STLM

5 TBATS

6 FASSTER

7 Comparisons

Automatic Forecasting Algorithms ETS 5

Exponential smoothing methodsSeasonal Component

Trend N A MComponent (None) (Additive) (Multiplicative)

N (None) (N,N) (N,A) (N,M)A (Additive) (A,N) (A,A) (A,M)Ad (Additive damped) (Ad,N) (Ad,A) (Ad,M)

(N,N): Simple exponential smoothing(A,N): Holt’s linear method(Ad,N): Additive damped trend method(A,A): Additive Holt-Winters’ method(A,M): Multiplicative Holt-Winters’ method(Ad,M): Damped multiplicative Holt-Winters’ method

There are also multiplicative trend methods (not recommended).Automatic Forecasting Algorithms ETS 6

Methods V ModelsExponential smoothing methods

Algorithms that return point forecasts.

Innovations state space modelsGenerate same point forecasts but can also generateforecast intervals.A stochastic (or random) data generating process thatcan generate an entire forecast distribution.Allow for “proper” model selection.ETS(Error,Trend,Seasonal):

Error = {A,M}Trend = {N,A,Ad}Seasonal = {N,A,M}.


Exponential smoothing methodsSeasonal Component


N (None) N,N N,A N,M

A (Additive) A,N A,A A,M

Ad (Additive damped) Ad,N Ad,A Ad,M


Exponential smoothing modelsSeasonal Component





General notation E T S : ExponenTial Smoothing↗ ↑ ↖

Error Trend SeasonalExamples:A,N,N: Simple exponential smoothing with additive errorsA,A,N: Holt’s linear method with additive errorsM,A,M: Multiplicative Holt-Winters’ method with multiplicative errors


Exponential smoothing modelsSeasonal Component





There are 9 separate exp. smoothing methods.Each can have an additive or multiplicative error, giving 18separate models.Only 15 models are numerically stable.Additive and multiplicative error models give same pointforecasts but different prediction intervals.All models can be written in innovations state space form.


Exponential smoothing modelsAdditive Error Seasonal Component


N (None) A,N,N A,N,A A,N,M

A (Additive) A,A,N A,A,A A,A,M

Ad (Additive damped) A,Ad,N A,Ad,A A,Ad,M

Multiplicative Error Seasonal ComponentTrend N A M

Component (None) (Additive) (Multiplicative)

N (None) M,N,N M,N,A M,N,M

A (Additive) M,A,N M,A,A M,A,M

Ad (Additive damped) M,Ad,N M,Ad,A M,Ad,MAutomatic Forecasting Algorithms ETS 11

ETS state space modelsxt−1

εt

yt

{}


State space modelxt = (level, slope, seasonal)


εt

yt

xt

{}




εt

yt

xt yt+1

εt+1

{}




εt

yt

xt yt+1

εt+1 xt+1

{}




εt

yt

xt yt+1

εt+1 xt+1 yt+2

εt+2

{}




εt

yt

xt yt+1

εt+1 xt+1 yt+2

εt+2 xt+2

{}




εt

yt

xt yt+1

εt+1 xt+1 yt+2

εt+2 xt+2 yt+3

εt+3

{}




εt

yt

xt yt+1

εt+1 xt+1 yt+2

εt+2 xt+2 yt+3

εt+3 xt+3

{}




εt

yt

xt yt+1

εt+1 xt+1 yt+2

εt+2 xt+2 yt+3

εt+3 xt+3 yt+4

εt+4

{}




εt

yt

xt yt+1

εt+1 xt+1 yt+2

εt+2 xt+2 yt+3

εt+3 xt+3 yt+4

εt+4

{}



EstimationCompute likelihood L fromε1, ε2, . . . , εT.Optimize L wrt model parameters.

ETS state space modelsLet xt = (`t, bt, st, st−1, . . . , st−m+1) and εt

iid∼ N(0, σ2).

yt = h(xt−1)︸︷︷︸ + k(xt−1)εt︸︷︷︸ Observation equationµt et

xt = f(xt−1) + g(xt−1)εt State equation

Additive errors:k(xt−1) = 1. yt = µt + εt.

Multiplicative errors:k(xt−1) = µt. yt = µt(1 + εt).εt = (yt − µt)/µt is relative error.


Innovations state space models

Estimation

L∗(θ, x0) = n log( n∑

t=1ε2t /k

2(xt−1))

+ 2n∑t=1

log |k(xt−1)|

= −2 log(Likelihood) + constant

Estimate parameters θ = (α, β, γ, φ) and initial statesx0 = (`0, b0, s0, s−1, . . . , s−m+1) by minimizing L∗.


Automatic forecasting

From Hyndman et al. (IJF, 2002):

Apply each model that is appropriate to the data.Optimize parameters and initial values using MLE.Select best method using AICc.Produce forecasts using best method.Obtain forecast intervals using underlying state spacemodel.

Method performed very well in M3 competition.


Example: Asian livestock

livestock %>% ets() %>% forecast() %>% autoplot()

300

400

500

600

1960 1980 2000

Time

.

level

80

95

Forecasts from ETS(M,A,N)


Example: drug sales

h02 %>% ets() %>% forecast() %>% autoplot()

0.3

0.6

0.9

1.2

1.5

1995 2000 2005 2010

Time

.

level

80

95

Forecasts from ETS(M,Ad,M)


ETS


ETS


www.OTexts.org/fpp2

ETS


Outline

1 Motivation

2 ETS

3 ARIMA models

4 STLM

5 TBATS

6 FASSTER

7 Comparisons

Automatic Forecasting Algorithms ARIMA models 19

ARIMA models

yt−1

yt−2

yt−3yt

Inputs Output


ARIMA models

yt−1

yt−2

yt−3

εt

yt

Inputs Output


Autoregression (AR) model

ARIMA models

yt−1

yt−2

yt−3

εt

εt−1

εt−2

yt

Inputs Output


Autoregression movingaverage (ARMA) model

ARIMA models

yt−1

yt−2

yt−3

εt

εt−1

εt−2

yt

Inputs Output



EstimationCompute likelihood L fromε1, ε2, . . . , εT.Use optimization algorithm tomaximize L.

ARIMA models

yt−1

yt−2

yt−3

εt

εt−1

εt−2

yt

Inputs Output



EstimationCompute likelihood L fromε1, ε2, . . . , εT.Use optimization algorithm tomaximize L.

ARIMA modelAutoregression movingaverage (ARMA) modelapplied to differences.

Auto ARIMAlivestock %>% auto.arima() %>% forecast() %>% autoplot()

300

400

500

1960 1980 2000

Time

.

level

80

95

Forecasts from ARIMA(0,1,0) with drift


Auto ARIMAh02 %>% auto.arima() %>% forecast() %>% autoplot()

0.3

0.6

0.9

1.2

1995 2000 2005 2010

Time

.

level

80

95

Forecasts from ARIMA(3,1,3)(0,1,1)[12]


How does auto.arima() work?A non-seasonal ARIMA process

φ(B)(1− B)dyt = c + θ(B)εtNeed to select appropriate orders p, q, d, and whether toinclude c.


Algorithm choices driven by forecast accuracy.

How does auto.arima() work?A non-seasonal ARIMA process

φ(B)(1− B)dyt = c + θ(B)εtNeed to select appropriate orders p, q, d, and whether toinclude c.

Hyndman & Khandakar (JSS, 2008) algorithm:Select no. differences d via KPSS unit root test.Select p, q, c by minimising AICc.Use stepwise search to traverse model space, startingwith a simple model and considering nearby variants.


Algorithm choices driven by forecast accuracy.

How does auto.arima() work?A seasonal ARIMA process

Φ(Bm)φ(B)(1− B)d(1− Bm)Dyt = c + Θ(Bm)θ(B)εtNeed to select appropriate orders p, q, d, P,Q,D, andwhether to include c.

Hyndman & Khandakar (JSS, 2008) algorithm:

Select no. differences d via KPSS unit root test.Select D using OCSB unit root test.Select p, q, P,Q, c by minimising AICc.Use stepwise search to traverse model space, startingwith a simple model and considering nearby variants.


Outline

1 Motivation

2 ETS

3 ARIMA models

4 STLM

5 TBATS

6 FASSTER

7 Comparisons

Automatic Forecasting Algorithms STLM 25

STL decompositionSTL: “Seasonal and Trend decomposition using Loess”,Very versatile and robust.Unlike X-12-ARIMA, STL will handle any type ofseasonality.Seasonal component allowed to change over time,and rate of change controlled by user.Smoothness of trend-cycle also controlled by user.Robust to outliersNot trading day or calendar adjustments.Only additive.Take logs to get multiplicative decomposition.Use Box-Cox transformations to get otherdecompositions.


STL decompositionfit <- stl(elecequip, s.window=5, robust=TRUE)autoplot(fit) +

ggtitle("STL decomposition of electrical equipment index")

data

tren

dse

ason

alre

mai

nder

2000 2005 2010

60

80

100

120

80

90

100

110

−20

−10

0

10

−20

−10

0

10

Time

STL decomposition of electrical equipment index


STL decompositionfit <- stl(elecequip, s.window="periodic", robust=TRUE)autoplot(fit) +

ggtitle("STL decomposition of electrical equipment index")

data

tren

dse

ason

alre

mai

nder

2000 2005 2010

60

80

100

120

80

90

100

110

−10

0

10

−10−5

05

10

Time

STL decomposition of electrical equipment index


Forecasting and decomposition

Forecast seasonal component using seasonal naivemethod.Forecast seasonally adjusted data using non-seasonaltime series method. E.g., ETS or ARIMA.Combine forecasts of seasonal component withforecasts of seasonally adjusted data to get forecastsof original data.


Electrical equipmentfit <- stl(elecequip, s.window=7)fit %>% seasadj() %>% ets() %>% forecast() %>% autoplot() + ylab("seasonally adjusted elecequip")

60

80

100

120

2000 2005 2010 2015

Time

seas

onal

ly a

djus

ted

elec

equi

p

level

80

95

Forecasts from ETS(A,Ad,N)


Electrical equipmentfit %>% forecast() %>%

autoplot()

50

75

100

125

2000 2005 2010 2015

Time

elec

equi

p level

80

95

Forecasts from STL + ETS(A,Ad,N)


Forecasting and decompositionelecequip %>% stlf() %>%

autoplot() + ylab('elecequip')

50

75

100

125

2000 2005 2010 2015

Time

elec

equi

p level

80

95

Forecasts from STL + ETS(A,Ad,N)


Decomposition and prediction intervals

It is common to take the prediction intervals from theseasonally adjusted forecasts and modify them withthe seasonal component.This ignores the uncertainty in the seasonalcomponent estimate.It also ignores the uncertainty in the future seasonalpattern.


Outline

1 Motivation

2 ETS

3 ARIMA models

4 STLM

5 TBATS

6 FASSTER

7 Comparisons

Automatic Forecasting Algorithms TBATS 34

Complex seasonality

7000

8000

9000

10000

1990 2000 2010 2020

Year

Tho

usan

ds o

f bar

rels

per

day

Gasoline data


Complex seasonality

7000

8000

9000

10000

1990 2000 2010 2020

Year

Tho

usan

ds o

f bar

rels

per

day

Forecasts from TBATS(1, {0,0}, −, {<52.18,9>})


Complex seasonality

7000

8000

9000

10000

1990 2000 2010 2020

Year

Tho

usan

ds o

f bar

rels

per

day

Forecasts from TBATS(1, {0,0}, −, {<52.18,9>})

blah


fit <- tbats(gasoline)fcast <- forecast(fit)autoplot(fcast)

Complex seasonality

100

200

300

400

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Week

Num

ber

of c

alls

in 5

min

ute

inte

rval

Number of calls to large American bank {7am..9pm}


Complex seasonality

100

200

300

400

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Week

Num

ber

of c

alls

in 5

min

ute

inte

rval

Forecasts from TBATS(1, {3,1}, 0.8, {<169,6>, <845,4>})


Complex seasonality

100

200

300

400

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Week

Num

ber

of c

alls

in 5

min

ute

inte

rval

Forecasts from TBATS(1, {3,1}, 0.8, {<169,6>, <845,4>})

blah


fit <- tbats(calls)fcast <- forecast(fit)autoplot(fcast)

TBATS model

yt = observation at time t

y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi

+ dt

`t = `t−1 + φbt−1 + αdtbt = (1− φ)b + φbt−1 + βdt

dt =p∑i=1φidt−i +

q∑j=1θjεt−j + εt

s(i)t =

ki∑j=1

s(i)j,t


s(i)j,t = s(i)

j,t−1 cosλ(i)j + s∗(i)

j,t−1 sinλ(i)j + γ(i)

1 dts(i)j,t = −s(i)

j,t−1 sinλ(i)j + s∗(i)

j,t−1 cosλ(i)j + γ(i)

2 dt

TBATS model


y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi

+ dt




s(i)t =

ki∑j=1

s(i)j,t


s(i)j,t = s(i)






2 dt

Box-Cox transformation

TBATS model


y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi

+ dt




s(i)t =

ki∑j=1

s(i)j,t


s(i)j,t = s(i)






2 dt


M seasonal periods

TBATS model


y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi

+ dt




s(i)t =

ki∑j=1

s(i)j,t


s(i)j,t = s(i)






2 dt


M seasonal periods

global and local trend

TBATS model


y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi

+ dt




s(i)t =

ki∑j=1

s(i)j,t


s(i)j,t = s(i)






2 dt


M seasonal periods


ARMA error

TBATS model


y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi

+ dt




s(i)t =

ki∑j=1

s(i)j,t


s(i)j,t = s(i)






2 dt


M seasonal periods


ARMA error

Fourier-like seasonal terms

TBATS model


y(ω)t =

(yωt − 1)/ω if ω 6= 0;

log yt if ω = 0.

y(ω)t = `t−1 + φbt−1 +

M∑i=1

s(i)t−mi

+ dt




s(i)t =

ki∑j=1

s(i)j,t


s(i)j,t = s(i)






2 dt


M seasonal periods


ARMA error

Fourier-like seasonal terms

TBATSTrigonometricBox-CoxARMATrendSeasonal

TBATS modelTBATSTrigonometric terms for seasonalityBox-Cox transformations for heterogeneityARMA errors for short-term dynamicsTrend (possibly damped)Seasonal (including multiple and non-integer periods)

Handles non-integer seasonality, multiple seasonalperiods.Entirely automatedPrediction intervals often too wideVery slow on long series


Outline

1 Motivation

2 ETS

3 ARIMA models

4 STLM

5 TBATS

6 FASSTER

7 Comparisons

Automatic Forecasting Algorithms FASSTER 43

Pedestrian counts


Pedestrian counts

0

1000

2000

3000

4000

Jul 2016 Oct 2016 Jan 2017 Apr 2017

Date

Ped

estr

ians

cou

nted

Hourly pedestrian traffic at Southern Cross Station

0

1000

2000

3000

4000

Apr 01 Apr 15 May 01 May 15 Jun 01

Date

Ped

estr

ians

cou

nted

Hourly pedestrian traffic at Southern Cross Station


Pedestrian counts

Weekday Weekend

00 A

M

06 A

M

12 P

M

18 P

M

00 A

M

00 A

M

06 A

M

12 P

M

18 P

M

00 A

M

0

1000

2000

3000

4000

Time

Tota

l ped

estr

ians

cou

nted

Seasonality in pedestrian traffic at Southern Cross Station


Switching StructureFASSTER extends current state-space approaches byswitching between states.Dynamic linear model

yt = Ftθt + vt, vt ∼ N (0,V)θt = Gθt−1 + wt, wt ∼ N (0,W)

Switch between two groups:

yt = It∈G1Ftθ(1)t + It∈G2Ftθ

(2)t + vt vt ∼ N (0,V)

Groups G1 and G2 define the switching rule (say weekdaysand weekends).


Application to pedestrian trafficThis series contains a switching daily pattern overweekdays and weekends. Each group can be modelledusing level and hourly seasonal states.

0

1000

2000

3000

4000

Jan 30 Feb 06 Feb 13 Feb 20

Date

Ped

estr

ians

cou

nted

Pedestrian Traffic at Southern Cross Station


Application to pedestrian traffic

An appropriate model can be constructed using switchingstates:

yt = It∈WeekdayFtθ(Weekday)t + It∈WeekendFtθ(Weekend)

t + vt

where Ftθ(i)t = `(i)

t + f(i)t and vt ∼ N (0,V)

`t is a level componentft is a seasonal component based on Fourier terms


FASSTER allows flexible use of:

seasonal factorsfourier seasonal termspolynomial trendsBoxCox transformationsexogenous regressorsARMA processesstate switching

General measurement equation

yt = F(0)t θ

(0)t +

k∑j=1

It∈GjF(j)t θ

(j)t + vt, vt ∼ N (0,V)

where k is the number of switching combinations.Automatic Forecasting Algorithms FASSTER 50

FASSTER and Dynamic Linear ModelsA FASSTER model can be represented as a time-varyingDLM.

yt = Ftθt + vt, vt ∼ N (0,V)θt = Gθt−1 + wt, wt ∼ N (0,W)

where θ0 ∼ N (m0, C0)

There are too many parameters to easily estimate.E.g., pedestrian model has 48 states:2 groups× (1 level + 23 fourier states).We use a “heuristic estimation” approach involvingonly two passes through the data.


Usage (Pedestrian Counts)SthCross_fasster_fit <- SthCross_Ex %>%

fasster(Hourly_Counts ~ DayType %S% (poly(1) + trig(24)))

0

1000

2000

3000

4000

0 100 200 300 400 500

Date

Ped

estr

ians

cou

nted

Series

Response

Fitted

Pedestrian Traffic at Southern Cross Station


Forecasts (Pedestrian counts)


Outline

1 Motivation

2 ETS

3 ARIMA models

4 STLM

5 TBATS

6 FASSTER

7 Comparisons

Automatic Forecasting Algorithms Comparisons 54

Pharmaceutical sales

0.50

0.75

1.00

1.25

1995 2000 2005 2010

Time

h02

Forecast

ARIMA

ETS

FASSTER

STLF

TBATS


R packages


https://github.com/earowang/tsibble

http://pkg.earo.me/sugrrants

https://github.com/mitchelloharawild/fasster

http://pkg.robjhyndman.com/forecast

http://pkg.earo.me/hts

Textbook


OTexts.org/fpp2

Automatic Forecasting Algorithms - Rob J. Hyndman · 2017-11-18 · Motivation Common in business...

Documents

Transcript of Automatic Forecasting Algorithms - Rob J. Hyndman · 2017-11-18 · Motivation Common in business...