Based on forecasts you've seen, what would you say is a "reasonable" ratio of the standard deviation of the forecast error to the standard deviation of a covariance-stationary series being forecast? ... It would be great if you can tell me "I'd consider x reasonable and y too high."The problem is that the premise underlying the question (namely, that there is such a "reasonable" value of the ratio r of innovation variance to unconditional variance) is false. That is, there's no small value c of r such that r<c means that we've done a good forecasting job. Equivalently, there's no large value c′ of the predictive R2 (R2=1−r2) such that R2>c′ means that we've done a good forecasting job. Instead, "good" c or c′ values depend critically on the dynamic nature of the series being forecast. Consider, for example, a covariance-stationary AR(1) process, yt=ϕyt−1+εt, where εt∼iid(0,σ2). The innovation variance is σ2 and the unconditional variance is σ2/(1−ϕ2), so the lower bound on r (and hence the upper bound on R2) depends entirely on ϕ and can be anywhere in the unit interval! This is an important lesson: "predictability" can (and does) differ greatly across economic series. For more than you ever wanted to know, see Diebold and Kilian (2001), "Measuring Predictability: Theory and Macroeconomic Applications".
Monday, May 4, 2015
Measuring Predictability
A friend writes the following. (I have edited very slightly for clarity.)
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.