Model based economic forecasting€¦ · I “learning the art or superforecasting”, cf, Tetlock...

Intro Optimality of model forecasts Vulnerability to structural breaks Conclusions References

Model based economic forecasting

Ragnar Nymoen

ECON 4160 Lecture 1324 November 2016


Dynamic econometric models and alternatives I

I HN Ch 21.I Forecasting is a time oriented activity. And adjustment of

economic variables is usually non-instantaneous.Imply that models have to be dynamic to be suited forforecasting

I Obvious today, but if we draw a time-line of macroeconomic models, a surprising long segment of that linewould be dominated by models where time played noessential role (ie, they were static models), cf. Granger andNewbold (1986) in spite of early offerings like Haavelmo(1940).

The alternatives to econometric models are:I Expert judgement

2/22


Dynamic econometric models and alternatives II

I Statistical time series modelsIn practice: few purists.

One advantage of using econometric models for forecasting isthat they can be seen as restricted statistical time series models,and it is easy to incorporate expert judgement in the modelbased forecasts, by the use of add-factors, aka intercept-correction.

In the following: Look a little at the “pure case’, it is easy tothink of a long list of ‘pros’ for model based forecasting inmacroeconomics

3/22


Advantages of forecasting with economic models I

1. Interpretability: In particular the clarification of initial andequilibrium condition provided by a model, makes themodel based forecast more interpretable than statisticalforecasts

2. Consistency with definitional relationships: For exampleforecasted total demand must equal total supply

3. Internally consistent multivariate forecasts. Forecasting from amacro model implies that the forecasting function of allvariables contains a common set of key parameters:Characteristic roots of the companion form.

4. Co-integration Cointegrating relationships also hold forforecasts, and give additional interpretability to initialconditions.

4/22


Advantages of forecasting with economic models II5. Forecasting and policy analysis with the same model. Policy

analysis and forecasting can be separately validated and ittakes super structural breaks to damage both forecasts, andto invalidate policy analysis.

6. Cost effective: There are fixed costs and variable cost ofestablishing an operational model, but marginal cost ofpreparing a “forecast round” can be very low, also foralternative future scenarios.

7. Contingency planning. Probabilistic forecasts can be usefulfor contingency planning, also when prediction intervalsare wide.

8. Optimality According to standard textbooks ineconometrics, model based forecasts are optimal: have thelowest MSFE, Mean Squared Forecast Error.

5/22


But optimality is derived from assumptions:

1. The forecast is conditioned on period T2. Perfect measurement of initial conditions;3. The model is linear in parameters and correctly specified.4. Disturbances for T + 1, T + 2,... are unpredictable based on

period T-information.5. Parameters are known (no estimation problem), up until T.6. The preference function of the forecaster is quadratic7. The structure of the economy remain relatively unchanged

in the forecast period (ie., parameter constancy)

If 1-6 are OK, the optimal forecast of YT+h, h = 1, 2, ..,H is theconditional expectation from the model: E(Yt+h |Model). It haslowest MSFE.

6/22


Example model

I Ct private consumption in period t, in fixed prices.I GDPt, St and Jt represent GDP, tax revenue and exogenous

expenditures (gross capital formation; and governmentconsumption)

I a - e are parametersI εCt, og εJt og εTt are mean-zero, constant variance

independent white-noise processes.

Ct = a + b(GDPt − St) + cCt−1 + εCt (1)St = d + eGDPt + εSt (2)

GDPt = Ct + Jt (3)Jt = J∗ + εJt (4)

7/22


Optimal forecasts, h-periods ahead, from period T

CT+h|T = C∗ + λh(CT − C∗) − 1 < λ < 1 (5)

GDPT+h|T = GDP∗ + λh(GDPT − GDP∗) (6)

JT+h|T = J∗, h = 1, 2, ..,H (7)

I 0 < λ < 1 under conventional assumptions aboutparameters b, c and e

I C∗,GDP∗ og J∗ are equilibrium values of Ct, GDPt and Jt.I The forecasts equilibrium correctI Could have said: expectation correct, since

C∗ = E(Ct) = µC etc8/22


That was the forecast. But what does the economydo?

I It also equilibrium corrects; But not always to the sameequilibrium as the forecast!

I In our example model, if J∗ → J∗∗ in period T + 1,T + 2, ...then⇒ C∗ → C∗∗ :

CT+h = C∗∗ + λh(CT − C∗∗) + eT+h

where eT+h is an ‘average’ of future disturbances. The forecasterrors become

CT+h − CT+h = dC∗ − λhdC∗ + eT+h (8)dC∗ = C∗∗ − C∗ (9)

Unbiased forecasts as long as the economy and the modelagrees about what the equilibrium is in the forecast period.

9/22


Forecasts for consumption (C) when economy andmodel agree

Prognose med estimerte koeff Prognose med kjente koeff

C C

100 101 102 103 104 105 106 107 108 109 110 111 112 113

250

251

252

253

254

255

256

257

258

259 Prognose med estimerte koeff Prognose med kjente koeff

C C

10/22


Structural break

I dC∗ “large” means that there is regime-shift in the forecastperiod

I Not one Black Swan (Taleb), but a whole flock (Hendry)I It is the location shifts that represent the real obstacle to

forecasting with our discipline modelsI Intermittent location-shifts also explain why “naive”

forecasts can do well in forecast contests.

11/22


’Location shift’ in the forecast periodA drop in investment after the preparation and publication ofthe forecast

Forecasts C

100 105 110

245.0

247.5

250.0

252.5

255.0

257.5 Forecasts C Forecasts S

100 105 110

157.5

160.0

162.5

165.0

Forecasts S

Forecasts J

100 105 110

95.0

97.5

100.0

Forecasts J Forecasts BNP

100 105 110

355

360

365

370Forecasts BNP

12/22


The model based forecast do not error-correctI Breaks that take place in period T + 1 or later will damage

the forecast that conditioned on period T informationI Of course, after one period, we will update the forecast, by

conditioning on period T + 1 informationI The break is now in the information set used for forecastingI But the forecast nevertheless glides towards the old

(pre-break) end-point:

CT+2+h|T+1 = C∗ + λh(CT+1 − C∗),

and not towards the new equilibrium C∗∗.I Model based forecasts are therefore vulnerable to breaks

that happen in the forecast period, and to breaks that havehappened before we make the forecast

13/22


A drop in investment before the preparation of theforecast

Forecasts C

105 110

245.0

247.5

250.0

252.5

255.0

257.5Forecasts C Forecasts S

105 110

157.5

160.0

162.5

165.0Forecasts S

Forecasts J

105 110

95.0

97.5

100.0

102.5 Forecasts J Forecasts BNP

105 110

355

360

365

370 Forecasts BNP

14/22


Implications for policy analysis?

I Exactly the same properties that make the model forecastsvulnerable to breaks makes is relevant for policy analysis:

I “What is the effect on C and GDP if there are shock toinvestments? ”

I Hence, models that fail in forecasting need not bediscredited as models for policy analysis Castle andHendry (2015)

I Invariance of derivative parameters (not intercetpts) withrespect to shocks to J is the relevant property for policyanalysis.

15/22


Implications for modeled based forecasting I

I When doing forecasting, the parameters that you care leastabout when estimating and testing, become the mainparameters of interest

I Namely: Constant terms that affect equilibrium means andautonomous growth rates

I When those parameters are estimated with little bias, andare stable over time, there are low costs, and severalbenefits from forecasting by the use of econometricdiscipline models

I But there is no “fix” (within current family of disciplinemodels) for breaks that occur after the forecast has beenprepared

16/22


Implications for modeled based forecasting II

I Hence, at the time of the break, and immediately after, willneed correction by model user and/or objective methodsfor correctly detecting breaks

I In the medium-term perspective, model surveillance andadaptation to known breaks certainly help.

Forecast, measure, revise. Repeat. It’s a never-endingprocess of incremental improvement that explains whyweather forecasts are good and slowly getting better.Tetlock and Gardner (2015)

17/22


Implications for modeled based forecasting IIIand, speaking about the practical need of model maintenance:

By the time a system has been designed to give explicitdisplay to a variable that has appeared to be important, theeconometrician may find that some new variable, formerlysubmerged in aggregation, is now important. ... Every twoor three years the model must be revised to keep it up to date.Klein (1962, p.269)

I Conversely, keeping a model specification unchanged, willquickly reduce the forecast performance, Nymoen (2002)

I An intriguing perspective is that the intermittent structuralbreaks that damage the forecasts of our linear(ized)equilibrium correction models, may be symptoms of adeeper weakness.

18/22


Implications for modeled based forecasting IV

I And that our models are not descriptive accurate enoughfor the complex dynamics of real world of economicssystem, cf. Buchanan (2013)

I In that perspective, the future of economic forecasting islinked to the future of economic modelling.

I Almost certainly too much simple equilibrium (correction)dynamics in current discipline models.

19/22


Implications for modeled based forecasting V

I Whether the road to improvement lies in the direction ofI complex dynamics, orI behavioural economics, orI “learning the art or superforecasting”, cf, Tetlock and

Gardner (2015), one does not know.I But for certain, it does not lie in playing down the

importance of economic forecasts and of their evaluations.

20/22


A big forecast evaluation project(from Tetlock (2005)

I VARs og ARDLs are thewinners (foxes)in thiscontest.

I Not the experts’(‘talking heads’)judgement (they arehedgehogs).

21/22


References I

Buchanan, M. (2013). Forecast. What Physics, Meteorology and the Natural Sciences Can Teach Us About Economics.Bloomsbury, London.

Castle, L., J and D. Hendry (2015). On Not Evaluating Economic Models By Forecast Outcomes. Instanbul UniversityJournal of the School of Business Administration, 40, 1–21.

Granger, C. W. J. and P. Newbold (1986). Forecasting Economic Time Series. Academic Press, San Diego.

Haavelmo, T. (1940). The Inadequacy of Testing Dynamic Theory by Comparing Theoretical Solutions and ObservedCycles. Econometrica, 8(4), 312–321.

Klein, L. R. (1962). An Introduction to Econometrics. A Non-Technical Treatment of Apllications and Methods Based onClassical Studies in the Field.. Prentice-Hall.

Nymoen, R. (2002). Faulty Watch Towers: Structural Models in Norwegian Monetary Policy Analysis. Tech. rep.,Department of Economics, University of Oslo.

Tetlock, P. (2005). Expert Political Judgement. Princeton Univeristy Press.

Tetlock, P. and D. Gardner (2015). Superforecasting. The Art and Science of Prediction. Crown Publishers, New York.

22/22

Model based economic forecasting€¦ · I “learning the art or superforecasting”, cf, Tetlock...

Documents

Transcript of Model based economic forecasting€¦ · I “learning the art or superforecasting”, cf, Tetlock...