FORECASTING AND MODEL SELECTION - IIT Kanpur and Forecasting Forecasting Methods Measuring the...
Transcript of FORECASTING AND MODEL SELECTION - IIT Kanpur and Forecasting Forecasting Methods Measuring the...
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
FORECASTING AND MODEL SELECTION
Anurag Prasad
Department of Mathematics and StatisticsIndian Institute of Technology Kanpur, India
REACH Symposium, March 15-18, 2008
1 Forecasting and Model Selection
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Outline
1 Modeling and Forecasting
2 Forecasting Methods
3 Measuring the Accuracy of Forecast
2 Forecasting and Model Selection
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Assumptions of Forecasting
1 Element of Uncertainty
2 Blind Spots
3 Change in Forecast Accuracy
3 Forecasting and Model Selection
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Framework of a Forecast System
No
Historical Data
Data
SpecificationModel
Checking
Diagnostic Estimation
Model ForecastGeneration
StabilityChecking
ForecastUpdation
NewObservations
Yes Yes
Model−Building Phase Forecasting Phase
No
Theory and/or
4 Forecasting and Model Selection
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Choice of a Particular Forecast Model
1 Degree of Accuracy Required
2 Cost of Producing Forecasts
3 Forecast Horizon
4 Degree of Complexity Required
5 Available Data
5 Forecasting and Model Selection
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Classification of Estimation Methods
1 Time Series Methods
2 Causal Methods
3 Judgemental Methods
6 Forecasting and Model Selection
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Time Series Methods
• Use historical data as a basis• Underlying patterns are fairly stable
1 Autoregressive Moving Average (ARMA)2 Exponential Smoothing3 Extrapolation4 Linear Prediction5 Trend Estimation6 Growth Curve7 Box-Jenkins Approach
7 Forecasting and Model Selection
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Causal Methods
• Belief that some other time series can be useful• Assumption that it is possible to identify the underlying factors
1 Regression Analysis* Linear Regression* Non-Linear Regression
2 Econometrics
8 Forecasting and Model Selection
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Judgemental Methods
• Incorporate intuitive judgements, opinions and probabilityestimates
1 Composite Forecasts2 Surveys3 Delphi Method4 Scenario Building5 Technology Forecasting6 Forecast by Analogy
9 Forecasting and Model Selection
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Forecast Error
For t = 1, . . . , N,y(t) : Actual value at period t ,y(t) : Forecast value at period t ;e(t) : Forecast error at period t ; e(t) = y(t) − y(t)
i
y(t )i
t it
y(t)
y(t )
^
10 Forecasting and Model Selection
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Graphical Measures of Forecast Accuracy
Plot of y(t) versus y(t)
• Keep the same scale for both the axes.• Departure of points from the 450 line through origin indicatesimperfect forecasts.
11 Forecasting and Model Selection
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Correct Model Form
y(t)
y(t)Incorrect Model Form
y(t)
y(t)
12 Forecasting and Model Selection
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Plot of e(t) versus t
• Reveals patterns of variability which the model has failed toexplain.• For a good model, the forecast errors should vary in ahorizontal band around zero.
13 Forecasting and Model Selection
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
0
tCorrect Model Form
0
tIncorrect Model Form
0
tIncorrect Model Form
0
e(t) e(t)
e(t) e(t)
tIncorrect Model Form
14 Forecasting and Model Selection
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Descriptive Measures of Forecast Accuracy
Descriptive Measures of Forecast Accuracy are used to...
1 Provide a single, easily interpreted measure of model’sreliability
2 Compare the accuracy of two different models3 Search for an optimal model4 Monitor a model’s performance
15 Forecasting and Model Selection
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Descriptive Measures of Forecast Accuracy
Mean Absolute Deviation (MAD)
MAD =�
|forecast error|number of forecasts =
� Nt=1 |e(t)|
N
Mean Square Error (MSE)
MSE =�
(forecast error)2
number of forecasts =� N
t=1 e(t)2
N
Root Mean Square Error (RMSE)
RMSE =√
MSE
16 Forecasting and Model Selection
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Descriptive Measures of Forecast Accuracy
Mean Absolute Percent Error (MAPE)
MAPE =�
|forecast error/actual value|number of forecasts .100%
=� N
t=1 |e(t)/y(t)|N .100%
Pearson’s Correlation Coefficient (r )between y(t) and y(t)
r =� N
t=1(y(t)−y)( �y(t)−¯�y)� � Nt=1(y(t)−y)2
� � Nt=1( �y(t)−¯�y)2
17 Forecasting and Model Selection
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Descriptive Measures of Forecast Accuracy
"No Change" model is : y(t + 1) = y(t)
Theil’s Inequality Coefficient (U)
U = RMSE("new" model)RMSE("no change" model)
U > 1 ⇒ worse than "no change" model
U = 1 ⇒ as good as "no change" model
U < 1 ⇒ better than "no change" model
18 Forecasting and Model Selection
Modeling and ForecastingForecasting Methods
Measuring the Accuracy of Forecast
Further Readings
Quantitative Forecasting Methods, N.R. Farnum and L.W.Stanton, 1989, PWS-KENT Publishing Co.
Statistical Methods for Forecasting, B. Abraham and J.Ledolter, 1983, John Wiley & Sons
Introduction to Time Series and Forecasting, P.J. Brockwelland R.A. Davis, 2002 (Second Edition) , Springer-Verlag
Time Series Analysis and Forecasting, R. Yaffee, 2000,Academic Press
19 Forecasting and Model Selection