More on Factor-Augmented VAR's (Principal Components Regression)

Here's a sampling of emails that I received on my recent "Factor-Augmented VAR" post.

Serena Ng at Columbia notes that her "Targeted Predictors" paper (with Jushan Bai) is motivated by considerations similar to those that motivate partial least squares (PLS). She also notes that she has a discussion of this in a Handbook of Forecasting overview paper (sections 4 and 5), and that it's not clear that PLS is systematically dominant. I look forward to reading the Handbook piece, which, embarrassingly, I have not yet done.

George Kapetanios at Queen Mary, University of London, echoes Serena's view. He sent his new paper, "Revisiting Useful Approaches to Data-Rich Macroeconomic Forecasting" (with Jan Groen), in the PLS-Kelly-Pruitt tradition but considering more general settings (e.g., weak factors) and considering alternative methods such as ridge regression. His upshot is that forecasting is of course complex and "best" procedures depend on a variety of settings and choices (and the "best" may not be PLS), but that in any event principal-component regression (PCR) appears robustly sub-optimal.

Not least, Frank DiTraglia at Penn sent some interesting links to the chemometrics literature, which prominently features PLS and has some interesting probabilistic perspectives on it.

So it's interesting. PCR appears robustly sub-optimal, but there's a feeling that PLS at least as traditionally implemented does not appear robustly dominant. Other emails, not reported in detail here, echoed that theme,

So much for the PCR vs. PLS issue. What about the PCR vs. ridge regression (RR) issue? Enter Paramveer Dhillon, a Penn Computer Science (machine learning) Ph.D. student, who sent his paper, "A Risk Comparison of Ordinary Least Squares vs Ridge Regression" (with Dean Foster, Sham Kakade and Lyle Ungar). Paramveer et al. show that PCR risk is always within a factor of four of RR risk, but that the converse is not true; that is, RR can be arbitrarily worse than PCR. So from a different perspective PCR suddenly looks appealing. (And from the Blogger-Abusing-His-Position-to-Pat-Himself-on-the-Back Department: Paramveer also notes that he enjoyed my Ph.D. time-series course, which he audited last year!)

[Finally, my friends, just in case you missed the weekend post I'll repeat it: Please don't hesitate to post comments. Instead what usually happens is that people email me directly, and I can't respond, and I feel bad that I can't respond, and the sender feels bad that I didn't respond, and most importantly, people who would benefit from reading the comment (and perhaps reading comments on the comment, or themselves commenting on the comment) never get to see it. A bad equilibrium all around.]