1

Exponential smoothing:The state of the art – Part II

Everette S. Gardner, Jr.

2

Exponential smoothing:The state of the art – Part II

History Methods Properties Method selection Model-fitting Inventory control Conclusions

3

Timeline of Operations Research (Gass, 2002)

1654 Expected value, B. Pascal1733 Normal distribution, A. de Moivre1763 Bayes Rule, T. Bayes 1788 Lagrangian multipliers, J. Lagrange 1795 Method of Least Squares, C. Gauss, A. Legendre1826 Solution of linear equations, C. Gauss1907 Markov chains, A. Markov1909 Queuing theory, A. Erlang1936 The term OR first used in British military applications1941 Transportation model, F. Hitchcock1942 U.K. Naval Operational Research, P. Blackett1943 Neural networks, W. McCulloch, W. Pitts1944 Game theory, J. von Neumann, O. Morgenstern 1944 Exponential smoothing, R. Brown

4

Exponential smoothing at work

“A depth charge has a magnificent laxative effect on a submariner.”

Lt. Sheldon H. Kinney, Commander, USS Bronstein (DE 189)

5

Forecast Profiles N A M

None Additive Multiplicative

N

None

A

Additive

DA

Damped Additive

M

Multiplicative

DM

Damped Multiplicative

6

Damped multiplicative trends (Taylor, 2002)

-

1,000

2,000

3,000

4,0001.00 0.95 0.90

0.85

Damping paramete

r

7

Variations on the standard methods Multivariate series (Pfefferman & Allen, 1989) Missing or irregular observations (Wright,1986) Irregular update intervals (Johnston, 1993) Planned discontinuities (Williams & Miller, 1999) Combined level/seasonal component (Snyder & Shami, 2001) Multiple seasonal cycles (Taylor, 2003) Fixed drift (Hyndman & Billah, 2003) Smooth transition exponential smoothing (Taylor, 2004) Renormalized seasonals (Archibald & Koehler, 2003) SSOE state-space equivalent methods (Hyndman et al., 2002)

8

Smoothing with a fixed drift (Hyndman & Billah, 2003)

Equivalent to the “Theta method”?(Assimakopoulos and Nikolopoulos, 2000)

How to do it Set drift equal to half the slope of a regression on time Then add a fixed drift to simple smoothing, or Set the trend parameter to zero in Holt’s linear trend

When to do it Unknown

9

Adaptive simple smoothing (Taylor, 2004)

Smooth transition exponential smoothing (STES) is the only adaptive method to demonstrate credible improved forecast accuracy

The adaptive parameter changes according to a logistic function of the errors

Model-fitting is necessary

10

Renormalization of seasonals Additive (Lawton, 1998)

Without renormalization Level and seasonals are biased Trend and forecasts are unbiased

Renormalization of seasonals alone Forecasts are biased unless renormalization is done

every period

Multiplicative (Archibald & Koehler, 2003) Competing renormalization methods give forecasts

different from each other and from unnormalized forecasts

11

Archibald & Koehler (2003) solution

Additive and multiplicative renormalization equations that give the same forecasts as standard equations

Cumulative renormalization correction factors for those who wish to keep the standard equations

12

Continental Airlines Domestic Yields

0.10

0.11

0.12

0.13

0.14

0.15

Jan-00 Jan-01 Jan-02 Jan-03 Jan-04 Jan-05

ModelRestarted

13

Standard vs. state-space methods Trend damping

Standard: Immediate State-space: Starting at 2 steps ahead

Multiplicative seasonality Standard: Seasonal component depends on level State-space: Independent components

Model fitting Standard: Minimize squared errors State-space: Minimize squared relative errors if

multiplicative errors are assumed.

14

Properties

Equivalent models Prediction intervals Robustness

15

Equivalent models Linear methods

ARIMA DLS regression Kernel regression (Gijbels et al.,1999; Taylor, 2004) MSOE state-space models (Harvey, 1984)

All methods SSOE state-space models (Ord et al.,1997)

16

Analytical prediction intervals Options

SSOE models (Hyndman et al., 2005) Model-free (Chatfield & Yar, 1991)

Empirical evidence None

17

Empirical prediction intervals Options

Chebyshev distribution (fitted errors) (Gardner, 1988) Quantile regression (fitted errors) (Taylor & Bunn, 1999) Parametric bootstrap (Snyder et al., 2002) Simulation from assumed model (Bowerman, O’Connell,

& Koehler, 2005)

Empirical evidence Limited, but encouraging

18

Robustness Many equivalent models for each method

(Chatfield et al., 2001; Koehler et al., 2001)

Simple ES performs well in many series that are not ARIMA (0,1,1) (Cogger,1973)

Aggregated series can often be approximated by ARIMA (0,1,1) (Rosanna & Seater, 1995)

19

Robustness (continued)

Exponentially declining weights are robust (Muth, 1960; Satchell & Timmerman, 1995)

Additive seasonal methods are not sensitive to the generating process (Chen,1997)

The damped trend includes numerous special cases (Gardner & McKenzie,1988)

20

Automatic forecasting with the damped additive trend

= .84

= .38 = 1.00

21

Summary of 66 empirical studies,1985-2005

Seasonal methods rarely used Damped trend rarely used Multiplicative trend never used Little attention to method selection But exponential smoothing was robust,

performing well in at least 58 studies

22

Method selection

Benchmarking Time series characteristics Expert systems Information criteria Operational benefits Identification vs. selection

23

Benchmarking in method selection

Methods should be compared to reasonable alternatives

Competing methods should use exactly the same information

Forecast comparisons should be genuinely out of sample

24

Method selection: Time series characteristics

Variances of differences (Gardner & McKenzie,1988)

Seemed a good idea at the time

Discriminant analysis (Shah,1997)

Considered only simple smoothing and a linear trend Should be tested with an exponential smoothing

framework

Regression-based performance index (Meade, 2000)

Considered every feasible time series model Should be tested with an exponential smoothing

framework

25

Method selection: Expert systems Rule-based forecasting

Original version (Collopy & Armstrong, 1992)

Automatic version (Vokurka et al., 1996)

Streamlined version (Adya et al., 2001)

Other rule-induction systems(Arinze,1994; Flores & Pearce, 2000)

Expert systems are no better than aggregate selection of the damped trend alone (Gardner, 1999)

26

Method selection: AICDamped trend vs. state-space models selected by AIC:

Average of all forecast horizons

8

10

12

14

16

18

20

111 1,001 M3 Ann. M3 Qtr. M3 Mon.

Damped trend

State-space

MAPE Asymmetric MAPE

27

Method selection:Empirical information criteria (EIC)

Strategy: Penalize the likelihood by linear and nonlinear functions of the number of parameters (Billah et al., 2005)

Evaluation: EIC superior to other information criteria, but results are not benchmarked

28

Method selection: Operational benefits

Forecasting determines inventory costs, service levels, and scheduling and staffing efficiency.

Research is limited because a model of the operating system is needed to project performance measures.

29

Method selection: Operational benefits (cont.) Manufacturing (Adshead & Price, 1987)

Producer of industrial fasteners (£4 million annual sales)

Costs: holding, stockout, overtime

U.S. Navy repair parts (Gardner, 1990) 50,000 inventory items Tradeoffs: Backorder delays vs. investment Savings: $30 million (7%) in investment

30

Average delay in filling backorders

Damped trend

Simple smoothing

Linear trend

Random walk

25

30

35

40

45

50

370 380 390 400 410 420 430

Inventory investment (millions)

Backo

rde

r d

ays

31

Inventory analysis: Packaging materials for snack-food manufacturer

Month$0

$500,000

$1,000,000

$1,500,000

$2,000,000

$2,500,000

Target maximum inventory based on damped trend Monthly

Usage

Actual Inventory from

subjective forecasts

Month

32

Method selection: Operational benefits (cont.) Electronics components (Flores et al., 1993)

967 inventory items Costs: holding cost vs. margin on lost sales

RAF repair parts (Eaves & Kingsman, 2004) 11,203 inventory items Tradeoffs: inventory investment vs. stockouts Savings: £285 million (14%) in investment

33

Forecasting for inventory control:Cumulative lead-time demand

SSOE models yield standard deviations of cumulative lead-time demand (Snyder et al., 2004)

Differences from traditional expressions (such as ) are significant

timeLeads

34

Standard deviation multipliers, α = 0.30

Lead time

0

1

2

3

4

5

2 3 4 5 6

TraditionalCorrect

35

The parametric bootstrap (Snyder et al., 2002) can estimate variances for: Any seasonal model Non-normal demands Intermittent demands Stochastic lead times

Forecasting for inventory control:Cumulative lead-time demand (cont.)

36

Forecasting for inventory control:Intermittent demand

Croston’s method (Croston, 1972)

Smoothed nonzero demandMean demand = Smoothed inter-arrival time

Bias correction (Eaves & Kingsman, 2004; Syntetos & Boylan, 2001, 2005)

Mean demand x (1 – α / 2)

37

Forecasting for inventory control: Intermittent demand (continued) There is no stochastic model for Croston’s

method (Shenstone & Hyndman, 2005) Many questionable variance expressions in the

literature The state-space model for intermittent series requires

a constant mean inter-arrival time (Snyder, 2002)

Why not aggregate the data to eliminate zeroes?

38

Progress in the state of the art, 1985-2005

Analytical variances are available for most methods through SSOE models.

Robust methods are available for multiplicative trends and adaptive simple smoothing.

Croston’s method has been corrected for bias. Confusion about renormalization of seasonals has

finally been resolved. There has been little progress in method selection. Much empirical work remains to be done.

39

Suggestions for research Refine the state-space framework

Add the damped multiplicative trend Damp all trends immediately Test alternative method selection procedures

Validate and compare method selection procedures Information criteria – Benchmark the EIC Discriminant analysis Regression-based performance index

40

Suggestions for research (continued) Develop guidelines for the following choices:

Damped additive vs. damped multiplicative trend Fixed vs. adaptive parameters in simple smoothing Fixed vs. smoothed trend in additive trend model Standard vs. state-space seasonal components Additive vs. multiplicative errors Analytical vs. empirical prediction intervals

41

Conclusion

“The challenge for future research is to establish some basis for choosing among these and other approaches to time series forecasting.” (Gardner,1985)