Based on forecasts you've seen, what would you say is a "reasonable" ratio of the standard deviation of the forecast error to the standard deviation of a covariance-stationary series being forecast? ... It would be great if you can tell me "I'd consider x reasonable and y too high."The problem is that the premise underlying the question (namely, that there is such a "reasonable" value of the ratio \(r\) of innovation variance to unconditional variance) is false. That is, there's no small value \(c\) of \(r\) such that \(r<c\) means that we've done a good forecasting job. Equivalently, there's no large value \(c'\) of the predictive \( R^2~ (R^2 = 1 - r^2) \) such that \(R^2 > c'\) means that we've done a good forecasting job. Instead, "good" \(c\) or \(c'\) values depend critically on the dynamic nature of the series being forecast. Consider, for example, a covariance-stationary AR(1) process, \(y_t = \phi y_{t-1} + \varepsilon_t\), where \(\varepsilon_t \sim iid (0, \sigma^2)\). The innovation variance is \(\sigma^2\) and the unconditional variance is \(\sigma^2 / (1 - \phi^2)\), so the lower bound on \(r\) (and hence the upper bound on \(R^2\)) depends entirely on \(\phi\) and can be anywhere in the unit interval! This is an important lesson: "predictability" can (and does) differ greatly across economic series. For more than you ever wanted to know, see Diebold and Kilian (2001), "Measuring Predictability: Theory and Macroeconomic Applications".
Monday, May 4, 2015
Measuring Predictability
A friend writes the following. (I have edited very slightly for clarity.)
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.