The difference between the forecasted value (F) and the actual value (A). By convention, the error is generally reported as F minus A. Forecast errors serve three important functions: (1) *The development of prediction intervals*. Ideally, the errors should be obtained from a test that closely resembles the actual forecasting situation. (2) *The selection (or weighting) of forecasting methods*. Thus, one can analyze a large set of forecasts and then select based on which method produced the more accurate forecasts. In such evaluations, the error term should be immune to the way the series is scaled (e.g., multiplying one of the series by 1,000 should not affect the accuracy rankings of various forecasting methods). Generally, the error measure should also be adjusted for the degree of difficulty in forecasting. Finally, the measure should not be overly influenced by outliers. The Mean Squared Error, which has been popular for years, should not be used for forecast comparisons because it is not independent of scale and it is unreliable compared to alternative measures. More appropriate measures include the APE (and the MdMAPE when summarizing across series) and the Relative Absolute Erros (and the MdRAE when summarizing across series). (3) *Refining forecasting models*, where the error measures should be sensitive to changes in the models being tested. Here, medians are less useful; the APE can be summarized by its mean (MAPE) and the RAE by its geometric mean (GmRAE). Armstrong and Collopy (1992a) provide empirical evidence to support these guidelines, and the measures are discussed in Armstrong (2001d).
- Armstrong, J. S. & F. Collopy (1992a), “Error
measures for generalizing about forecasting methods: Empirical comparisons,”
*International Journal of Forecasting*, 8, 69-80. (Full
text)
- Armstrong, J. S. (2001d), “Evaluating forecasting
methods,” in J. S. Armstrong (ed.),
*Principles of Forecasting.* Norwell,
MA: Kluwer Academic Press.