A GUIDE TO BOX-JENKINS MODELINGBy George C. S. Wang

Describes in simple language howto use Box-Jenkins models forforecasting ... the key requirementof Box-Jenkins modeling is thattime series is either stationary orcan he transformed into one ... themost difficult part in this type ofmodeling is the identification of amodel.

George Box and Gwilyni Jenkinsdeveloped a statistical approachfor time series modeling. Time

series models developed on the basis oftheir approach are called Box-Jenkinsmodels, also known as ARIMA models. Atime series can be defined as a sequence ofdata observed over time.

ARIMA models are univariate, thatis, they are based on a single time seriesvariable. Box and Jenkins have alsodeveloped procedures for multivarialemodeling. However, in practice, even theirunivariate approach, sometimes, is not aswell understood as the classic regressionmethod. The objective of this article isto describe the basics of univariate Box-Jenkins models in simple and laymanterms.

UNIVARIATE MODELING

The purpose of univariate modeling isto establish a relationship between thepresent value of a time series and its pastvalues so that forecasts can be made onthe basis of the past values alone.

Stationary Time Series: The firstrequirement for univariate Box-Jenkinsmodeling is that the time series data to bemodeled are either stationary or can be

transformed into one. We can define that astationary time series has a constant meanand has no trend overtime. A plot of thedata is usually enough to see if the data arestationary. In practice, few time series canmeet this condition, but as long as the datacan be transformed into a stationary series,a Box-Jenkins model can be developed.

THE MODELING PROCESS

Box-Jenkins modeling of a stationarytime series involves the following foursteps:

GEORGE C. S. WANGDr. Wang is currently an independent

consultant, specializing in statisticalmodeling and business forecasting.Formerly, he was Forecast Manager atConsolidated Edison Company of NewYork, and was responsible for forecastmodeling and forecasting. He also hasserved as Company witness and gavetestimony in regulatory proceedings. Heis the co-author of the book. RegressionAnalysis: Modeling and Forecasting. Hereceived his M.B.A. and Ph.D. degreesfrom New York University.

1. Model identification2. Model estimation3. Diagnostic Checking4. Forecasting

The four steps are sinîilar to thoserequired for linear regression except thatStep I isalittlemoreinvolved. Box-Jenkinsuses a statistical procedure to identifya model, which can be confusing. Theother three steps are quite straightforward.Let's first discuss the mechanics of Step1, model identification, which we woulddo in great detail. Then we will use anexample to illustrate the whole modelingprocess.

MODEL IDENTIFICATION

ARIMA stands for Autoregressive-Integrated-Moving Average. The letter"!"(Integrated) indicates that the modelingtime series has been transformed into astationary time series. ARIMA representsthree difTerent types of models: It can bean AR (autoregressive) model, or a MA(moving average) model, or an ARMAwhich includes both AR and MA terms.Notice that we have dropped the "1" fromARIMA for simplicity. Let's briefly definethese three model forms.

AR Model: An AR model looks likea linear regression model except thatin a regression model the dependentvariable and its independent variablesare different, whereas in an AR modelthe independent variables are simplythe time-lagged values of the dependentvariable, so it is autoregressive. An ARmodel can include diflcrent numbers ot"autoregressive ternis. If an AR modelincludes only one autoregressive letm.it is an AR ( 1 ) model; we can also haveAR (2), AR (3), etc. An AR model can belinear or nonlinear.

THE JOURNAL OF BUSINESS FORECASTINQ SPRING 2008 19

MA Model: A MA model is a weightedmoving average of a fixed number offorecast errors produced in the past, soit is called moving average. Unlike thetraditional moving average, the weightsin a MA are not equal and do not sum upto I. In a traditional moving average, theweight assigned to each of the n values tobe averaged equals to 1 /n; the n weights areequal and add up to 1. In a MA, the numberof terms for the model and the weight foreach term are statistically determined bythe pattern of the data; the weights arenot equal and do not add up to I. Usually,in a MA. the most recent value carries alarger weight than the more distant values,For a stationary time series, one may useits mean or the immediate past value as aforecast for the next future period. Eachforecast will produce a forecast error. Ifthe errors so produced in the past exhibitany pattern, we can develop a MA model.Notice that these forecast errors are notobserved values; they are generatedvalues. All MA models, such as MA (1),MA (2). MA (3), are nonlinear.

ARMA Model: An ARMA modelrequires both AR and MA terms. Givena stationary time series, we must firstidentify an appropriate model form. Is itan AR, or a MA or an ARMA? How manyterms do we need in the identified model?To answer these questions, we need tocalculate the autocorrelation ftinction andthe partial autocorrelation function of theseries.

What are Autocorrelation Function(ACF) and Partial AutocorrelationFunction (PACF)? Without going into themathematics, ACF values fall between -1and +1 calculated from the time series atditïerent lags to measure the significanceof correlations between the presentobservation and the past observations, andto determine how far back in time (i.e., ofhow many time-lags) are they correlated.

PACF values are the coefficients of alinear regression of the time series usingits lagged values as independent variables.When the regression includes only oneindependent variable of one-period lag.

FIGURE ITHEORETICAL ACF AND PACF CORRELOGRAMS

la. ACF Ib. PACF

0.8

0.6

0.4

0.2

1 2 3 4 5

Time Lags

0.8

0.6

0.4

0.2

1 2 3 4 5

Time Lags

2a. ACF 2b. PACF

0.4

0.2

0

-(1.2

-(1.4

-0.6

tJ L 7 s

Time Lags

0

-0.2

-0.4

-0.6

-0.8

3 4 5 6 7 8

Time Lags

3a. ACF 3b. PACF

0.4

0.2

0

-0.2

-0.4

-0.6

1 2.

ti;

Time Lags

0

-0.2

-0.4

-O.fi

-0.8

1 2 3 • 5 6 7 « 9

Time Lags

the coefficient of the independent variableis called first order partial autocorrelationfunction; when a second term of two-period lag is added to the regression, thecoefficient of the second term is calledthe second order partial autocorrelationfunction, etc. The values of PACF willalso fall between -1 and +1 if the timeseries is stationary.

How do we use the pair of ACF andPACF functions to identify an appropriatemodel? A plot of the pair will provideus with a good indication of what typeof model we want to entertain. The plotof a pair of ACF and PACF is called a

correlogram. Figure 1 shows three pairs oftheoretical ACF and PACF correlograms.

In modeling, if the actual correlogramlooks like one of these three theoreticalcorrelograms, in which the ACF dimin-ishes quickly and the PACF has only onelarge spike, we will choose an AR (1)model for the data. The " I " in parenthesisindicates that the AR model needs onlyone autoregressive term, and the model isan AR of order 1.

Notice that the ACF patterns in 2a and 3aare the same, but the large PACF spike in2b occurs at lag 1, whereas in 3b, it occurs

20 THE JOURNAL OF BUSINESS FORECASTINQ SPRING 2008

at lag 4. Although both correlogramssuggest an AR ( I ) model for the data, the2a and 2b pattern indicates that the oneautoregressive term in the model is of lag1; but the 3a and 3b pattern indicates thatthe one autoregressive term in the modelis of lag 4. If this lag 4 term is to representseasonality of period 4, we wilt denote thismodel as SAR (4) or AR (4^) to distinguishit from an AR (4) model, which includesfour autoregressive terms.

Suppose that in Figure 1, ACF andPACF exchange their patterns, that is, thepatterns of PACF look like those of theACF and the patterns of ACF look Hke thePACF having only one large spike, thenwe will choose a MA (I) model. Supposeagain that the PACF in each pair looks thesame as the ACF, and then we will try anARMA(1, 1).

So far we have described the simplestAR, MA, and ARMA models. Models ofhigher order can be so identified, of course,with difierent patterns of correlograms.Let's use an example to demonstrate whatwe have just discussed.

An Example

Table 1 shows the quarterly electricdemand in New York City from the firstquarter of 1995 through the fourth quarterof 2005. The demand is a time series.The data have been modified to simplifycalculations. Columns (2) and (4) show theoriginal quarterly demand data. Columns(6) and (8) show the quarterly differenceddata.

Stationarity: Is the demand seriesstationary? Figure 2 is a plot of theoriginal electric demand data in Columns(2) and (4) of Table 1. The plot clearlyshows that the demand data are quarterlyseasonal trending upward; consequently,the mean of the data will change overtime. As defined above, this time series isnot stationary.

Since the data are quarterly seasonal, oneway to transform the data into a stationaryseries is to perform a four-quarter seasonal

Year&Qt.(1)

9501

9502

9503

9504

9601

9602

9603

9604

9701

9702

9703

9704

9801

9802

9803

9804

9901

9902

9903

9904

0001

0002

TABLE 1QUARTERLY ELECTRIC

Original Data

SalesY,(2)

22.91

20.63

28.85

22.97

23.39

20.65

30.02

23.13

23.51

22.99

32.61

23.28

23.97

21.48

27.39

23.75

24.81

21.51

33.20

23.68

25.37

22.36

\ ear &Qt.(3)

0003

0004

0101

0102

0103

0194

0201

0202

0203

0204

0301

0302

0303

0304

0401

0402

0403

0404

0501

0502

0503

0504

Sales

(4)

33.36

23.50

24.95

22.22

34.81

24.64

26.21

23.45

31.85

25.28

25,76

22,88

34,02

25,80

25,91

24.07

36,60

26.43

27.08

24,99

41,29

26,69

Year&Qt.(5)

9501

9502

9503

9504

9601

9602

9603

9604

9701

9702

9703

9704

9801

9802

9803

9804

9901

9902

9903

9904

0001

0002

DEMAND

Difftrcnccd Dala

Salesy=Y-Y,^"' (6)

0.48

0.02

1.17

0.16

0.12

2.34

2.59

0,15

0.46

-1.51

-5.22

0,47

0.84

0,03

5,81

-0,07

0.56

0,85

Year&Qt.(7)

0003

0004

0101

0102

0103

0194

0201

0202

0203

0204

0301

0302

0303

0304

0401

0402

0403

0404

0501

0502

0503

0504

Salesy,=Y-Y,^

(8)

0,16

-0.18

-0.42

-0.14

1,45

1,14

L26

1.23

-2.96

0.64

-0,45

-0.57

2,17

0,52

0.15

1,19

2.58

0.63

1.17

0.92

4,69

0.26

differencing in the following manner:

Let Yj be the original data point ofquarter t in Table I; let t = 9601, Y,,,„, =23.39, and let (t-4) = 9501, Y , , - 22'.9I.The quarterly differenced value y ^ Y- Y|^; data in Columns (6) and (8) werecalculated as follows:

V =Y - Y =^23 39 - 22 91=048

Similarly,

y,,„,= 20.65-20.63-0.02

y,,,,,,= 30.02-28.85 =1.17

The differenced values so calculated aregiven in Columns (6) and (8) of Table 1 and

plotted in Figure 3, Notice that, originally.the data base has 44 data points; the firstfour points were lost in ditTerencing, andthere are 40 points left for modeling.

After differencing, has the series becomestationary? Figure 3 shows that seasonaldilTercncing has eliminated the trend fromthe data, and the mean ofthe data will notchange over time. The series has becomestationary, and we are ready to develop anARMA model.

Modet Identification: As discussedbefore, the tools for identifying a goodmodel for a stationary time series are itsACF and PACF. ACF and PACF are thetwo statistical terms used in Step I ofARMA modeling. When we go through the

THE JOURNAL OF BUSINESS FORECASTING, SPRING 2008 21

calculations, we can easily find that theyare analogous to correlation coelficientand partial correlation coefficient inmultiple linear regression analysis.

The ACF and PACF values are giveiîin Table 2. which were calculated forten lags. Let's demonstrate manuallyhow to calculate the ACF and PACF oflag one.

In Table 1. Columns (6) and (8). wehave 40 differenced data points, so n =40. The differenced data has a mean (u) ^0.62.

Calculation of ACF of Lag I: Thecalculation of ACF is analogous to thecalculation of correlation coefficient. Onthe basis of data given in Table 1. ihe ACFoflag I is calculated below;ACFüflongerlags can be calculated similarly.

ACF of lag I -

Auto - cov aricmce

Variance

Auto-covariance of lag 1 =

[(0.48-0.62) (0.02-0.62) +40

(0.02-0.62) (1.17-0.62)

+ ...+ (4.69-0.62) (0.26-0.62)] - 8.53

Variance = — [(0.48-0.62)-+ (0.02-0.62)^40

+ ...+ (0.26-0.62)-]= 118.27

8.53ACF of lag 1 = - 0.072.

^ 118.27

Calculation of PACF of Lag 1: In Table2, the PACF of lag I also equals 0.072.We can use EXCEL regression add-Íns toregress y on y and obtain.

45

40

35

30

25

20

IS

10

FIGURE 2PLOT OF ORIGINAL DATA

10 15 20 25

Quarter

30 45 50

FIGURE 3PLOT OF DIFFERENCED DEMAND DATA

9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39

Lag

1

2

3

4

5

ACFAND

ACF

0.072

0.010.045

-0.396

-0,177

TABLE 2PACF AT DIFFERENT LAGS

PAC0.072

0.005

0.045

-0.406

-0.137

Lag6

7

8

9

10

ACF0.012

-0.05 1

0.148

0.122

0.029

PAC0.041

-0.003

0.015

-0.013

0.025

22 THE JOURNAL OF BUSINESS FORECASTING, SPRING 2008

As defined before, the coefficient, 0.072,of y I is the PACF of lag 1. Adding y , tothis regression equation, we will get thePACF of lag 2. etc.

The correlogram for the ACF andPACF, based on the data given in Table 2,is shown in Figure 4.

Does this correlogram look like one ofthe three sets of correlograms in Figure1? The ACF in Figure 4 is quite similarto those in 2a and 3a in Figure I, but thePACF here seems to look different fromPACF 2b and 3b in Figure 1. However,of the 10 bars in the PACF chart, there isonly one large spike at lag 4. If we ignorethe nine smaller bars in this chart, then itbecomes similar to chart 3b in Figure 1.We have said that the patterns of charts3a and 3b in Figure 1 suggested an AR(4^) model, and then the two charts inFigure 4 also suggest an AR (4^) model asfollows:

y = c + 0y _ + a . -. Í1 )

Although we denote Equation (1) as AR(4^), it is an AR ( 1 ) model in the sense thatit has only one autoregressive term, whichmodels seasonality of period 4.

MODEL ESTIMATION ANDDIAGNOSTIC CHECKING

The next two steps are for estimation ofthe model coefficients and diagnosticallychecking the goodness of fit. These twosteps are usually done together.

Estimation: In fact, most of identifiedARMA models are nonlinear requiring anonlinearestimation procedure. Only somesimple AR models are linear and can beestimated with the Ordinary Least Squares(OLS) procedure. For either procedure, thecriterion for getting the best estimates ofcoefficients is the same, that is, to minimizethe sum of the squared errors.

Equation ( 1 ) is clearly a linear regression

FIGURE 4CORRELOGRAM OF ACF AND PACF

0.2-,

0 -

-0.2-

0.4-

0.6-

ACF

I -

1Time

16 7 8

Lags

1.9 1

PACF

0.2

0

0.2

0.4

0.6

1 2 3 I Í 6 7 8 9 I

Time l.

equation, where c is the constant term, (j) isthe coefficient of y ^ and a is the residual.We have estimated the model with bothprocedures as follows:

y, = 0.863 - 0. + a, -.(2)

The residual, a,, in Equation (2) isexpected to be zero in forecasting. Theinterested readers can use the data givenin Columns (6) and (8) of Table 1, anduse EXCEL to verify the estimatedcoefficients in Equation (2). !f one woulduse a nonlinear procedure, it will takethree iterations to get Equation (2).

Suppose that the identified model is aMA (4^) as follow:

Equation (3) is nonlinear because a isnot observable, and it must be generated.We have to use the nonlinear least squaresprocedure to produce a (the historicforecast errors) before we can iterativelyestimate coefficient 0.

Notice that Box and Jenkins used thebackward shift operator B in their analysisvery extensively. For example, theydenoted y ^ = By,, y^, ^ B-y, y ^ = B^y.etc. In this article, we have avoided theuse of B.

Diagnostic Checking: Regardless whatestimation procedure is used in modeling,the criteria for testing the goodness of fitare the same. We use the R' to measure

the degree of correlation between thedependent variable and the independentvariables; we use the t-statistics to testthe significance of the coefficients and thestandard error to measure how closely themodel fits the data.

We also need to check the stability ofthe estimated model. For an AR ( 1 ) model,we require that -l<(t)<l. An AR (2) modelhas two coefficients, <[), and tj),, we requirethat:

Equation (2) has a coefficient of-0.4675, which falls between-1 and l.Themodel is stable. If these conditions are notmet, either because ihe time series is notstationary requiring more transformation,or because the model was not properlyidentified.

FORECASTING

Equation (2) is our model for forecasting,but we want to forecast the demand Y, notthe differenced value y . Therefore, wemust transform the model from the y fonnto the Y, form. Recall that y, ^ V, - Y ^ andy, _,= Yj _,-Y, j^. Equation (2) becomes,

0.863-0.465 . . . ( 4 )

Notice that we have dropped the a tenuin Equation (4) because in forecasting, ais assumed to be zero. Re-arranging termsin Equation (4), we obtain.

THE JOURNAL OF BUSINESS FORECASTING SPRING 2008 23

Y - 0.863+ {1 -0.465) Y+ 0.465Y,,

+0.465Y. . . . ( 5 )

Suppose that we want to forecast thedemand for the first quarter of 2006,according to Equation (5), we need thedemand data for the first quarters of 2005and 2004. From Table I. Y, ,„, - 27.08 andY„„, = 25.91, then.

= 0.863

Y,,^,,-0.863 + 0.535x27.08+ 0.465 «25.9! -27.40

Forecast accuracy can be similarlyevaluated as in linear regressioti.

CONCLUDING REMARKS

It is obvious that the tnost difficult stepin ARIMA modeling is Step 1, the modelidentification. Once we get a handle on Step1, the other three steps are quite similarto those in linear regression. Althoughthe calculations of the ACF and PACFand the nonlinear estimation procedurelook complicated and tedious, computersoftware is available to do these jobs.

In the example, the data base originallyincluded 44 points; we lost 4 points indifferencing. The identified model has aterm of lag 4; therefore, only 36 data pointswere available for mode! estimation. Thisis the reason why in ARIMA modeling,we need a relatively large sample size toaccommodate data loss due to ditïerencingand lagged structure of the model. •

UPCOMING EVENTSDemand Planning & Forecasting:

Best Practices ConferenceLas Vegas. NV - April 30- May 2, 2008

For Information:Call/Contact

Institute of Business Forecasting &Planning

Ph. 516.504.7576, Email lnfo@ibforg

BE A MEMBER OF THEINSTITUTE OF BUSINESS FORECASTING & PLANNING

Benefits include:•Journal of Business ForecastingComplimentary for active IBF Members, eachissue gives you a host of jargon-free articleson how to obtain, recognize, and use goodforecasts written in an easy-to-understandstyle for business executives and managers.Plus, it provides new. practical forecastingideas to help you make vital decisions aboutsales, capital outlays, credit, plant expansion,financial planning, budgeting, inventorycontrol, production scheduling and marketingstrategies. A one-year subscription includes 4issues. Most of the articles are written for andby practicing forecasters.

•Jonrnal of Business Forecasting PastArticles NEW! Active Members will nowhave FULL access to all Journal of BusinessForecasting articles since inception. Withactive IBF Membership, you will have theability to download unlimited .pdf files ofarticles based on your set search criteria.This way you will have access to research atyour fingertips! You can access hundreds ofarticles representing a multitude of industries,companies, and topics including demandplanning and supply chain management.This access will give you a step ahead inimproving your forecasting performance.There is no other body of knowledge whichis as extensive as this one and is gearedprimarily towards forecasting practitioners.

•Benchmarking Research Reports Ourbenchmarking reports will provide youwith understanding of key metrics and howyour company measures up. The ultimateoutcome of these studies is to gain a solidunderstanding of the "best in class" metricsmost companies are achieving. Researchincludes; benchmarks of forecasting errors,forecasting software/systems, forecastingsalary, and more. These indepth studiesof topics are based on various surveys offorecasting professionals from IBF events aswell as from other sources.

•Knowledge & Action Templates Ourgrowing online knowledge base includes keyissues and information on forecasting. This

co\ cTi. issues sucli as "How to Win the Supportof Top Management for Forecasting," "Howto Select Forecasting Software/ Systems." andmore. Plus, you will have access to electroniccopies of the latest journal. Moreover, youwill also have access to our Action Templates,ready to use. Currently, they include; (1)How to calculate forecast error? (2) How tocalculate how much money you will save byreducing specific amount of error? (3) How tocalculate safety stocks (forthcoming).

•Events & Training (Discounts available)IBF Conferences and Tutorials can raiseyour forecasting accuracy to new levels. Getstep-by-step training, hear case studies fromforecasting professionals working in wellknown companies, see demos of the latestsoftware packages and systems, networkand make long lasting connections with yourforecasting peers, and more. Our events arerun in Europe, Asia, as well as in the U.S.A.Plus, we also offer online events through ourWebinar series.

Join us at an IBF event today! For a fullschedule of our upcoming events andtestimonials, visit us online: www.ibf org.

•In-House Training Seminars (Discountsavailable) Bring the IBF to your workplace.Enjoy the convenience of a professionallydeveloped forecasting training programfor your staff at a location of your choiceanywhere in the world. Gain knowledge andhands-on training that can be put to use rightaway. Companies that recently had In-HouseTraining include: GAP. Cadbury, Wachovia.Wyeth, GlaxoSmithKline, Nike, Molson.and more. Call us for further details today!Discounts are applicable for CorporateMembers.

•Forecasting Books (Disconnts available)Our books are geared toward helpingprofessionals learn, process, interpret, andimplement Business Forecasting information.In addition, if you miss one of our conferences,we öfter manuals that detail each speaker'spresentation from all our conferences.

Individual IVIembership

S250 Domestic:, $300 Foreign

Corporate Membership(8 People Maximum)

$1800 Domestic, S2000 Foreign

Call 516.504.7576 or visit us on the vwsnw.ibf.org to sign upl

28 THE JOURNAL OF BUSINESS FORECASTING, SPRING 2008