Sunday, October 11, 2015

On Forecast Intervals "too Wide to be Useful"

I keep hearing people say things like this or that forecast interval is "too wide to be useful." 

In general, equating "wide" intervals with "useless" intervals is nonsense. A good (useful) forecast interval is one that's correctly conditionally calibrated; see Christoffersen (International Economic Review, 1998). If a correctly-conditionally-calibrated interval is wide, then so be it. If conditional risk is truly high, then a wide interval is appropriate and desirable.

[Note well:  The relevant calibration concept is conditional. It's not enough for a forecast interval to be merely correctly unconditionally calibrated, which means that an allegedly x percent interval actually winds up containing the realization x percent of the time. That's necessary, but not sufficient, for correct conditional calibration. Again, see Christoffersen.]

Of course all this holds as well for density forecasts.  
Whether a density forecast is "good" has nothing to do with its dispersion. Rather, in precise parallel to interval forecasts, a good density forecast is one that's correctly conditionally calibrated; see Diebold, Gunther and Tay (International Economic Review, 1998).