Forecasting Accuracy of Commodity Futures Against Random Walk

8/13/2019 Forecasting Accuracy of Commodity Futures Against Random Walk

1/22

Can Futures Outperform a Random Walk in Commodity

Markets?

Andrew Krog*

University of Michigan

April 17, 2013

Abstract

The idea that commodity futures are efficient forecasters of future spot prices has never

been proven definitively. Nonetheless, it is a widely held belief among market participants. This

paper explores the theory and tests the forecasting ability of the futures price against a simple

random walk model, that has proven difficult to beat in many forecasting applications. Six highly

traded commodities (corn, crude oil, gasoline, gold, natural gas, silver) from three commodity

classes are included to identify differences in forecasting ability across classes. Recursive

forecasts are constructed for 1-, 3-, 6-, 9-, and 12-month horizons using a spread regression to

obtain mean square prediction errors, which are compared between the regression and random

walk model. The modified Diebold-Mariano statistic is used to test the null hypothesis of equal

predictive power. This statistic is then bootstrapped using a nave block bootstrap to find 95%

confidence intervals. The results show no statistical difference between the random walk and

spread regression, although the MSPE is lower for the random walk for the majority of cases.

Keywords: Recursive Forecasts, Diebold-Mariano Statistic, Block Bootstrap

* Department of Economics, University of Michigan, Ann Arbor, MI 48109. Email: [email protected]


2/22

1

1 Introduction

The question of whether or not commodity futures prices are good forecasters of future

spot prices is one that has been frequently addressed yet never answered definitively.

Commodity price forecasts are widely used by policymakers as well as businesses, so

understanding the reliability of these forecasts is crucial to making optimal policy and

investment decisions. Intuitively, a forecasting method should outperform a random walk

benchmark in order to be considered for policy of investment decision making. A frequently

used method, which is the one addressed in this paper, is the futures price.

Futures prices are often looked to as a measure of expected future spot price due to the

assumption that all information available to the market is reflected in the current future price.

However, futures prices have shown differing levels of reliability depending on commodity class

and forecast horizon. This paper will test the forecasting ability of futures prices against a

random walk at 1-, 3-, 6-, 9-, and 12-month horizons for three classes of commodities: softs,

metals, and energy. The six commodities analyzed are corn, crude oil, gasoline, gold, natural gas,

and silver.

Forecasting ability will be determined by constructing recursive forecasts regressing the

change in spot price on the basis (difference between futures price and spot) and an intercept

using least squares. The Diebold-Mariano test statistic will then be used to determine whether the

difference in the predicted mean squared errors for random walk and spread regression is

significantly greater than zero (one-sided test). Since we are considering nested models with

parameter uncertainty we will use the variance of the spread regression predicted mean square

errors to determine the standard error. Furthermore, it is necessary to bootstrap this statistic since

the distribution is not asymptotically N(0,1), a mistake frequently made in past research using the

Diebold-Mariano statistic.


3/22

2

2 Data

The data begins in December 1993 and is collected monthly through December 2012

except for gasoline which only has futures data starting in 2006. Prices are taken in a three day

window between the 14thand 16thof every month depending on which are trading days.

Furthermore, the month used for each horizon depends on the last trading day on the contract.

Specifications vary by commodity, and the last trading day for contracts differs widely. Trading

for gold and silver ends on the 3rdlast business day of the month; corn ends on the business day

prior to the 15thof delivery month; natural gas terminates trading three days prior to the first day

of delivery month, gasoline on the last day of month before delivery, and crude oil on the 3rd

business day prior to the 25thof the month preceding delivery.

Table 1. Commodity Contract Details

Commodity Currency/Units Spot Futures Exchange

Corn /Bu Corn No. 2 Yellow CBOT

Crude Oil $/bbl WTI Spot Cushing NYMEX

Gasoline /gal Gasoline RBOB NYMEX

Gold $/Troy oz. Gold Bullion London Bullion Market COMEX

Natural Gas $/MMBtu Natural Gas-Henry Hub NYMEX

Silver $/Troy oz. Silver Fix LBM Cash COMEX

The futures price data is collected from Bloomberg using all available contracts for each

commodity. The energy and metals commodities have contracts expiring in each of the twelve

months while corn only offers five. The spot prices were collected from Datastream over the

same period. The future spot prices are taken for the expiration month of the futures contract and

the day is chosen to match the horizon.


4/22

3

3 Methods

3.1 Forecasting Methods

The model that will be tested for forecasting efficiency of the futures price is a spread

regression used to predict the change in spot price. This model will be tested against a random

walk benchmark.

The future spot price in the random walk model at the h month horizon is simply the

current spot price plus the sum of errors from next period through period h. Since the expected

value of the future period errors is zero, the random walk model implies a no change forecast and

the current spot price is the expected future spot price.

(1) t+h | t= StFor the spread regression, least squares is used to regress the log difference of future and

current spot price on the log difference of current futures price at horizon h and current spot

price.

(2) st+h- st= + *(fthst) + t+h(3) t+h | t=fthst

The expectation is that =1 and =0 (equation 3) under the null hypothesis of forecast efficiency,

but the intercept term is allowed to be nonzero in the regression.

A key assumption in this least squares construction of the regression is that the policy

makers or investors using the forecasts have a quadratic loss function which may not be the case.

An absolute loss function does not penalize large deviations from the estimated price

disproportionately whereas the least squares method does. Furthermore, the loss function may

not even be symmetric as both the quadratic and absolute loss functions are. One example would

be policy makers concerned with gas prices rising above a certain level but not concerned with

price drops. However, since it is impossible to know the loss function of all commodity price


5/22

4

forecasters, the quadratic loss function is a reasonable assumption. Symmetry of the loss function

implies risk neutrality and the quadratic nature is used mainly for convenience.

3.2 Recursive Pseudo Forecasts

The two models are used to generate pseudo out-of-sample forecasts to simulate

forecasting with the data in real-time. The initial forecasting period of 24 months is large enough

to give reliable estimates of the regression parameters and small enough to give reliable

estimates of the PMSE with the remaining sample. For each period from May 1996 (the 25th

month of data) until November 2012, the spread regression is re-estimated using the least squares

method described in section 3.1 on all past data. The regression at each period is used to generate

forecasts of the change in spot price at the corresponding horizon, which be compared with the

actual future spot price to construct the PMSE for the model. For the random walk, the PMSE is

simply obtained by using the spot price and h-month horizon future spot.

3.3 Diebold-Mariano Pseudo Out-of-Sample Test for Predictive Accuracy

In order to test for differences in predictive accuracy between the spread regression and

random walk model, the Diebold-Mariano test statistic is employed. The original statistic

supplied by Diebold and Mariano is:

DM =

1

-u0t

-u1t ar 1

-u0t

-u1t ~ N(0,1)

The notation is as follows: and are the recursive pseudo out-of-sampleforecast errors from the null model and alternative model respectively, T is the total sample size,

and R is the initial recursive window.

The spread regression is a nested model of the random walk since the models are

equialent when the parameters are zero. herefore, under the null hypothesis of =0, the


6/22

5

ariance of the difference in PMSEs will conerge to zero and the statistic will go to infinity.

The solution proposed by McCracken uses the variance of the PMSE of the alternative model,

which must be non-zero (Clark & McCracken, 1999). The new statistic is therefore:

DM =1

- u0t -u1t ar u1t

~ F

However, the distribution of test statistic is no longer asymptotically N(0,1) but rather an

unknown distribution F that must be bootstrapped.

3.4 Bootstrapping the distribution of the modified Diebold-Mariano statistic

Since the distribution F is unknown, nonparametric bootstrapping methods are used. A

nave block bootstrap is chosen due to autocorrelation in the prediction errors that can be seen in

the autocorrelation plots in the appendix. Nave block bootstraps allow serial correlation to be

retained in the resampling by selecting blocks whose length is increased as sample size increases.

Furthermore, we do not need to use a blocks of blocks bootstrap because the statistic of interest

is symmetric since it involves only a mean and variance. The bootstrap replication is therefore:

{yt*

t=1

= (1*,

*, , r*) where s/l

The length of the original sample is T, the number of blocks used to resample is s, and block

length is l. Also, yt*denotes the difference in squared prediction error between random walk and

spread regression at time t ( 0t -u1t ). For gasoline, l is chosen to be 5 since the sample size isvery short; for all other commodities, 15 is used. To construct the Diebold-Mariano statistic for

each of the 20,000 trials, first a simple mean function is used:

y *= yt=1 t*

Then this mean is divided by the variance of u1t using Newey-West standard errors:


7/22

6

y *

aru1t

Once the vector of 20,000 simulated statistics is constructed, 95% confidence intervals are found

by calculating the 2.5 and 97.5 percentiles. The null hypothesis is that there is no difference in

forecasting efficiency between the random walk and spread regression. The alternative is that

they do not have the same forecasting efficiency. If the calculated modified Diebold-Mariano test

statistic is outside of the 95% confidence interval then the null hypothesis is rejected at the 5%

level.

3.5 Possible Issues with Proposed Methods

It should be noted that there are multiple ambiguities in the above methods. First of all,

the confidence interval is sensitive to the choice of block length, and there is no general rule for

choosing it. The goal is to preserve any serial correlation in the data, so as sample size increases,

bootstrap block length will also increase. However, there is no obvious reason for choosing

specific numbers in this case. Another issue is the choice of lag for HAC standard errors. The

autocorrelation plots were observed to help with the decision, but these plots are by no means

definitive. This is less of a problem since the same standard errors are included in the calculation

of the bootstrapped distribution.

4 Results

The results show that the spread regression using the futures price had a lower MSPE in

only five of the thirty total cases for six commodities at five different horizons. The only cases

where the random walk has a larger MSPE are corn 3- and 6-month horizons as well as gasoline

3-, 6-, and 12-month horizons. None of these five are significant at the 5% level (Diebold

Mariano statistics and 95% confidence intervals are included in Appendix 1). However, even

though random walk has the lower MSPE in each of the other 25 cases, none of those are

statistically significant at the 5% level. Although there are no statistically significant results, the


8/22

7

fact that the MSPE is lower for the random walk in the majority of cases across three categories

of highly traded commodities shows that the futures price used in the spread regression is not an

ideal measure of the expected spot price.

Table 2. Mean Square Prediction Errors

Commodity Model 1-month 3-month 6-month 9-month 12-month

Corn RW 1486.5 3112.6 7669.5 11187 12775

Spread 1500.3 3030.9 7643 11594 13498

Crude Oil RW 9.1 80.1 257.8 348.2 377.8

Spread 29.6 126.2 288 360 404.7

Gasoline RW 541.8 1162 2210.7 2761 2969.1

Spread 588.5 1157.8 2126.1 2825 2964

Gold RW 1694.7 3763.9 8579.3 13705 20405

Spread 1703.2 3826.7 9049.9 14984 21840

Natural Gas RW .6476 1.9426 3.5552 4.5541 5.0215

Spread .6584 2.2415 4.1123 5.4413 5.6826

Silver RW 2.3 5.7 12.4 19.6 25.4

Spread 2.4 5.8 12.6 20.2 25.7

One explanation for this, as noted in Alquist & Kilian (2010), is that the difference

between the futures price and spot price is highly variable and may tend to fluctuate around the

current spot price. This is found to also be the case here although to a lesser extent. The

variability is not so large that it made the random walk a statistically significant more efficient

forecaster, although it does.

Another problem with the spread regression is that the bias and MAPE are very close to

those of the random walk for most horizons and commodities. Since the spread is higher variancethan the spot price, it needs to make up for this by being less biased, but that does not appear to

be the case here. One notable exception was corn for which the spread was significantly less

biased than the spot price. This led to lower mean square prediction errors for two horizons.

Furthermore, forecasts for all commodities were biased toward overestimating the future spot

price except for corn and the spread regression for natural gas.


9/22

8

5 Conclusion

As the results show, using the futures price in a spread regression does not provide a

better forecast for future spot price than a simple random walk model that uses a no-change

forecast. This has important implications for policy makers and investors using futures markets

to forecast or hedge. The assumption of futures prices as an efficient forecasting measure does

not seem to hold across any of the three commodity classes. There was no significant evidence

that the current spot price is a preferred method although it did have lower MSPE for nearly all

commodities and horizons. Further research might explore alternative uses of the futures price in

forecasting, particularly for the commodities in which it was continually overestimating the

future spot.


10/22

9

References

Alquist, R., & Kilian, L. (2010). What Do We Learn from the Price of Crude Oil Futures?Journal of

Applied Econometrics, 25(4), 539-573.

Bloomberg L.P. (2013, April 10). Commodity Futures Historical Contracts.

Clark, T. E., & McCracken, M. W. (1999). Tests of equal forecast accuracy and encompassing for nested

models.Journal of Econometrics, 105(1), 85-110.

Datastream International Commodity Spot Price Database (2013, April 10). Datastream International.

Diebold, F. X., & Mariano, R. S. (1995). Comparing Predictive Accuracy.Journal of Business &

Economic Statistics, 13(3), 253-263.

CME Group (2013, April 10). Corn Futures Contract Specifications.Retrieved from CME Group:

http://www.cmegroup.com/trading/agricultural/grain-and-oilseed/corn_contract_specifications.html

CME Group (2013, April 10). Gold Futures Contract Specifications.Retrieved from CME Group:

http://www.cmegroup.com/trading/metals/precious/gold_contract_specifications.html

CME Group (2013, April 10).Henry Hub Natural Gas Futures Contract Specifications.Retrieved from

CME Group: http://www.cmegroup.com/trading/energy/natural-gas/natural-

gas_contract_specifications.html

CME Group (2013, April 10).Light Sweet Crude Oil (WTI) Futures Contract Specifications.Retrieved

from CME Group: http://www.cmegroup.com/trading/energy/crude-oil/light-sweet-

crude_contract_specifications.html

CME Group (2013, April 10).RBOB Gasoline Futures Contract Specifications.Retrieved from CME

Group: http://www.cmegroup.com/trading/energy/refined-products/rbob-

gasoline_contract_specifications.html

CME Group (2013, April 10). Silver Futures Contract Specifications.Retrieved from CME Group:

http://www.cmegroup.com/trading/metals/precious/silver_contract_specifications.html

Kilian, L. (1999). Exchange Rates and Monetary Fundamentals: What Do We Learn from Long-Horizon

Regressions?Journal of Applied Econometrics, 14(5), 491-510.


11/22

10

Appendix 1Tables

Bias


Corn RW -3.37 3.51 20.25 23.48 34.50

Spread -3.87 -6.60 0.40 2.07 10.66

Crude Oil RW 0.43 1.26 2.58 3.61 4.69

Spread 0.43 1.03 2.32 3.56 4.92

Gasoline RW 3.46 13.66 27.99 35.83 43.25

Spread 6.13 21.36 34.28 39.21 45.56

Gold RW 6.14 19.56 38.52 57.52 78.64

Spread 6.38 19.69 37.54 55.05 81.09

Natural Gas RW 0.01 0.01 0.02 0.02 0.04Spread -0.01 -0.08 -0.08 -0.11 -0.07

Silver RW 0.12 0.40 0.78 1.13 1.57

Spread 0.09 0.35 0.67 0.99 1.58

Mean Absolute Prediction Error


Corn RW 24.57 38.39 59.11 70.35 79.39

Spread 24.55 38.67 59.18 72.36 82.56

Crude Oil RW 3.68 6.72 10.20 11.95 13.62

Spread 2.02 5.64 9.48 11.62 13.08

Gasoline RW 19.79 24.63 34.20 44.74 46.43

Spread 18.45 25.64 37.31 44.35 47.62

Gold RW 23.35 38.55 63.17 84.16 100.68

Spread 23.18 38.01 61.24 80.59 97.16

Natural Gas RW 0.54 1.03 1.33 1.55 1.62

Spread 0.54 0.97 1.26 1.48 1.66

Silver RW 0.80 1.28 1.97 2.54 2.87

Spread 0.77 1.27 1.93 2.47 2.82


12/22

11

Diebold-Mariano Statistics and Bootstrapped 95% Confidence Intervals

Corn

Horizon D-M Statistic Bootstrapped 95% Confidence Interval

1-month 4.20 [ 0.67, 11.44 ]3-month -12.68 [ -55.27, 26.86 ]

6-month -2.13 [ -46.50, 90.80 ]

9-month 22.61 [ -39.28, 127.92 ]

12-month 32.13 [ -41.35, 142.20 ]

Crude Oil


1-month 41.78 [ 12.45, 87.21 ]3-month 33.46 [ 2.13, 87.70 ]

6-month 12.87 [ .36, 30.59 ]

9-month 4.60 [ -.56, 10.99 ]

12-month 8.61 [ -.96, 21.71 ]

Gasoline


1-month 17.26 [ -4.0, 25.10 ]

3-month -0.69 [ -73.82, 43.75 ]6-month -9.25 [ -77.80, 11.61 ]

9-month 10.65 [ -42,14, 22.90 ]

12-month -0.67 [ -26.82, 19.50 ]

Natural Gas


1-month 0.18 [ -0.06, 0.38 ]

3-month 1.83 [ -0.52, 5.36]

6-month 2.00 [ -1.33, 6.99 ]

9-month 2.69 [ -0.17, 7.26 ]

12-month 1.64 [ -1.13, 5.67 ]


13/22

12

Gold


1-month 3.29 [ -.98, 10.50 ]

3-month 9.15 [ -0.10, 29.77 ]

6-month 37.71 [ 7.71, 85.73 ]9-month 57.09 [ 20.17, 159.45 ]

12-month 49.34 [ 18.26, 147.51 ]

Silver


1-month 1.34 [ -0.08, 2.11 ]

3-month 0.17 [ -0.19, 0.48 ]

6-month 0.40 [ -0.35, 1.38 ]

9-month 0.77 [ -0.23, 2.27]

12-month 0.36 [ -1.22, 1.48 ]


14/22

13

Appendix 2Figures

Autocorrelation Function Plots

Corn

Crude Oil

0 5 10 15 20 25 30-0.5

0

0.5

1

1-Month Horizon

0 5 10 15 20 25 30-0.5

0

0.5

1

3-Month Horizon

0 5 10 15 20 25 30-0.5

0

0.5

1

6-Month Horizon

0 5 10 15 20 25 30-0.5

0

0.5

1

9-Month Horizon

0 5 10 15 20 25 30-0.5

0

0.5

1

12-Month Horizon

0 10 20 30-1

0

1

1-Month Horizon

0 10 20 30-1

0

1

3-Month Horizon

0 10 20 30-1

0

1

6-Month Horizon

0 10 20 30-1

0

1

9-Month Horizon

0 10 20 30-1

0

1

12-Month Horizon


15/22

14

Gasoline

Gold

0 10 20 30-1

0

1

1-Month Horizon

0 10 20 30-1

0

1

3-Month Horizon

0 10 20 30-1

0

1

6-Month Horizon

0 10 20 30-1

0

1

9-Month Horizon

0 10 20 30-1

0

1

12-Month Horizon

0 10 20 30-1

0

1

1-Month Horizon

0 10 20 30-1

0

1

3-Month Horizon

0 10 20 30

-1

0

1

6-Month Horizon

0 10 20 30

-1

0

1

9-Month Horizon

0 10 20 300

0.5

1

12-Month Horizon


16/22

15

Natural Gas

Silver

0 10 20 30-1

0

1

1-Month Horizon

0 10 20 30-1

0

1

3-Month Horizon

0 10 20 30-1

0

1

6-Month Horizon

0 10 20 30-1

0

1

9-Month Horizon

0 10 20 30-1

0

1

12-Month Horizon

0 10 20 30-1

0

1

1-Month Horizon

0 10 20 30-1

0

1

3-Month Horizon

0 10 20 30-1

0

1

6-Month Horizon

0 10 20 30-1

0

1

9-Month Horizon

0 10 20 30-1

0

1

12-Month Horizon


17/22

16

Futures Prices

1994 1996 1998 2000 2002 2004 2006 2008 2010 2012

20

40

60

80

100

120

140

Crude Oil Futures Price for all Horizons

Date

Price($/bbl)

1-month

3-month

6-month

9-month

12-month

2007 2007.5 2008 2008.5 2009 2009.5 2010 2010.5 2011 2011.5 2012

150

200

250

300

350

Gasoline Futures Price for all Horizons

Date

Price(cents/gal)

1-month

3-month

6-month

9-month

12-month


18/22

17

1994 1996 1998 2000 2002 2004 2006 2008 2010 2012

400

600

800

1000

1200

1400

1600

1800

Gold Futures Price for all Horizons

Year

Price($/Troyoz.)

1-month

3-month

6-month

9-month

12-month

1994 1996 1998 2000 2002 2004 2006 2008 2010 2012

5

10

15

20

25

30

35

40

Silver Futures Price at all Horizons

Year

Price($/Troyoz.)

1-month

3-month

6-month

9-month

12-month


19/22

18

1994 1996 1998 2000 2002 2004 2006 2008 2010 2012

2

4

6

8

10

12

14

Natural Gas Futures Price for all Horizons

Year

Price($/MMBtu)

1-month

3-month

6-month

9-month

12-month

1994 1996 1998 2000 2002 2004 2006 2008 2010 2012

200

300

400

500

600

700

Corn Futures Price for all Horizons

Year

Price(cents/bu)

1-month

3-month

6-month

9-month

12-month


20/22

19

Spreads

1995 2000 2005 2010-0.05

0

0.05

0.1

1-month1995 2000 2005 2010

-0.2

-0.1

0

0.1

0.2

Corn Spread for All Horizons

3-month1995 2000 2005 2010

-0.3

-0.2

-0.1

0

0.1

0.2

6-month

1995 2000 2005 2010

-0.2

-0.1

0

0.1

0.2

0.3

9-month1995 2000 2005 2010

-0.2

-0.1

0

0.1

0.2

0.3

0.4

12-month

1995 2000 2005 2010

-0.2

-0.1

0

0.1

1-month1995 2000 2005 2010

-0.2

-0.1

0

0.1

Crude Oil Spread for All Horizons

3-month1995 2000 2005 2010

-0.2

-0.1

0

0.1

0.2

6-month

1995 2000 2005 2010-0.3

-0.2

-0.1

0

0.1

0.2

9-month1995 2000 2005 2010

-0.3

-0.2

-0.1

0

0.1

0.2

12-month


21/22

20

2007 2008 2009 2010 2011 2012-0.3

-0.2

-0.1

0

0.1

1-month2007 2008 2009 2010 2011 2012

-0.2

-0.1

0

0.1

0.2

Gasoline Spread for All Horizons

3-month2007 2008 2009 2010 2011 2012

-0.3

-0.2

-0.1

0

0.1

0.2

6-month

2007 2008 2009 2010 2011 2012

-0.3

-0.2

-0.1

0

0.1

0.2

9-month2007 2008 2009 2010 2011 2012

-0.2

-0.1

0

0.1

0.2

12-month

1995 2000 2005 2010

-0.01

-0.005

0

0.005

0.01

0.015

1-month1995 2000 2005 2010

0

0.01

0.02

0.03

Gold Spread for All Horizons

3-month1995 2000 2005 2010

0

0.01

0.02

0.03

0.04

0.05

0.06

6-month

1995 2000 2005 2010

0

0.02

0.04

0.06

0.08

9-month1995 2000 2005 2010

-0.15

-0.1

-0.05

0

0.05

0.1

0.15

0.2

12-month


22/22

21

1995 2000 2005 2010

-0.5

0

0.5

1-month1995 2000 2005 2010

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

Natural Gas Spread for All Horizons

3-month1995 2000 2005 2010

-0.4

-0.2

0

0.2

0.4

6-month

1995 2000 2005 2010-0.5

0

0.5

9-month1995 2000 2005 2010

-0.4

-0.2

0

0.2

0.4

12-month

1995 2000 2005 2010-0.06

-0.04

-0.02

0

0.02

0.04

1-month1995 2000 2005 2010

-0.06

-0.04

-0.02

0

0.02

0.04

0.06

Silver Spread for All Horizons

3-month1995 2000 2005 2010

-0.04

-0.02

0

0.02

0.04

0.06

0.08

6-month

1995 2000 2005 2010

0

0.05

0.1

9-month1995 2000 2005 2010

-0.2

0

0.2

0.4

12-month

Forecasting Accuracy of Commodity Futures Against Random Walk

Documents

Transcript of Forecasting Accuracy of Commodity Futures Against Random Walk