150 Research on Forecasting / International Journal of Forecasting 13 (1997) 149-153

forecasts, and the reverse was true when the environment had low predictability. However, the most accurate forecasts, in both predictabili- ty conditions, were obtained when subjects had access to both database and statistical model support. It seems that pattern matching enabled subjects to take into account the variable missing from the statistical model, while the availability of the model prevented over reliance on case histories, which was particularly valuable when predictability was low.

Overall, this is a well-organized and insightful paper. However, a section reviewing how the paper built upon our existing knowledge of judgmental forecasting would have been wel- come. The last couple of decades have seen the development of a sizable literature in this area, yet little of this research was referenced. We might also question some of the performance measures used on the experimental data. Corre- lations of forecasts and actuals ignore biases; error measures based on actuals minus forecasts are influenced, in part, by the ability of fore- casters to predict the artificially created noise. Nevertheless, in the light of the rapid advance of information technology and the increased availa- bility of data, this paper has important practical implications for designers of DSS.

Paul Goodwin [Stephen J. Hoch, The Wharton School, Uni- versity of Pennsylvania, Philadelphia, PA 19104, USA]

P I I S0169-2070(96)00722-4

T.L. Kastens and G.W. Brester, 1996, Model selection and forecasting ability of theory-con- strained food demand systems, American Jour- nal of Agricultural Economics, 78, 301-312.

Econometricians frequently estimate systems of demand equations by imposing restrictions de- rived from economic theory. For example, they

restrict parameter values so that compensated, marginal cross-price effects will be equal. Much of the time, statistical testing rejects the validity of the restrictions. Should theoretical restrictions still be used? Rarely does the researcher check the out-of-sample forecasting performance of a model to see how economic consequences com- pare with statistical significance.

Kastens and Brester compared the conditional forecasting performance of nine econometric estimations. Forecasts were conditional, since actual current price and income values and lagged quantity values were used to predict current quantity consumed. Data were from highly aggregated US sources. They consisted of annual price and quantity indexes for six food groups (for example, one group is red meat, fish and poultry) and disposable personal income from 1923 to 1992. Twenty-five observations were used in estimation, with re-estimation each year so that forecasts were available from 1949 to 1992 (44 one-year-ahead forecasts).

The authors estimated three demand system models: the Rotterdam system, a first-differ- enced linear approximation to the Almost Ideal Demand System (ALIDS), and a first-differ- enced double log system. Each model was esti- mated in two forms: with parameters unre- stricted and with theoretical restrictions (adding up, homogeneity and symmetry) imposed. The first two models are similar in specification, except for the dependent variable. Both are derived from consumer theory so there is no prior reason to favor one over the other. A few studies have estimated both models and made a choice based on how well each model conforms to theory for the given data set. No clear winner has emerged. Both models have been widely used to estimate demand systems, in particular, ALIDS specifications. The double log system is a set of Cobb-Douglas functions in first differ- ences. Because it is simpler and less flexible than the first two models, economists would expect the third model to be least accurate. It only conforms to consumer theory with severe param- eter restrictions (so the authors dropped the adding-up restrictions in their restricted estima-

Research on Forecasting ! International Journal of Forecasting 13 (1997) 149-153 151

tions). Conditional quantity forecasts were calcu- lated directly from these results.

Results can also be presented as price and income elasticities. The final three quantity fore- casts were based on these elasticities and on actual one-period changes in prices and income. Elasticities were calculated from the restricted parameter set of estimations. With the double log form, the coefficients are themselves elastici- ty estimates. With the other two systems, an elasticity formula contains coefficients and with- in-sample means of variables. Quantity forecasts were calculated from the following formula. The predicted percentage change in quantity is the summation over all prices of the actual one- period percentage change in a price multiplied by its elasticity, plus the actual one-period per- centage change in income multiplied by its elas- ticity. Calculating mean elasticities, then using them with actual changes in prices and income to predict changes in income, is the simplest predic- tion formula.

On the basis of statistical significance, the restrictions imposed by economic theory were rejected for all models. Out-of-sample forecasts were ranked by root mean square error (RMSE), both nominally, by counting successes in all possible paired comparisons, and statisti- cally, by counting only a significantly smaller RMSE as success. Statistical significance was tested using the method proposed by Ashley et al. (1980), which operates as follows. Given a series of forecast errors from method 1, with larger RMSE than a series from method 2, create a series of differences, by subtracting method 2 errors from method 1 errors, and a series of their sums. Calculate the deviations of the series of sums about its mean. Regress the series of differences on the series of deviations and a constant term. If both coefficients are positive, then when they are independently dis- tributed (as they are here, see Brandt and Bessler, 1983), their distribution under the null hypothesis of zero parameters is an F-distribu- tion with actual size one-fourth of nominal size (e.g. an F-statistic at the 20% level is significant at the 5% level). If one coefficient is negative

and insignificant, use a one-sided t-test on the positive coefficient to test the null hypothesis. If either coefficient is negative and significant, then the test for equality of RMSEs is inconclusive.

Elasticity-based forecasts were more accurate than restricted model forecasts, which were more accurate than unrestricted model forecasts. At least for these data, better forecasts result from imposing restrictions based on economic theory, even when the restrictions are statistically in- valid. Forecasts from the simpler and less flexible double-log system were better than those from the Rotterdm system, which were better than those from the ALIDS.

Overall, simpler models were more accurate. No comparisons were made with univariate models, and a fair comparison would require the one-step-ahead forecasts of six price and one income variables to show that econometric models could outperform univariate ones.

References

Ashley, R., C.W.J. Granger and R. Schmalensee, 1980, Advertising and aggregate consumption: An Analysis of causality, Econornetrica 48, 1149-1167.

Brandt, J.A. and D.A. Bessler, 1983, Price forecasting and evaluation: An application in agriculture, Journal of Fore- casting 2, 237-248.

Geoff Allen [Terry L. Kastens, Assistant Professor, Depart- ment of Agricultural Economics, Kansas State University, Manhattan, KS 66506, USA]

Pll s0169-2070(96)00723-6

R. Fildes and S. Makridakis, 1995, The impact o f empirical accuracy studies on time series analysis and forecasting, International Statistical Review, 63, 289-308.

Statisticians spend much time on the develop- ment- of methods for time-series forecasting. We