Tuesday, May 6, 2014

Predictive Modeling, Causal Inference, and Imbens-Rubin (Among Others)

When most people (including me) say predictive modeling, they mean non-causal predictive modeling, i.e., addressing questions of "What will likely happen if the gears keep grinding in the usual way"? Examples are ubiquitous and tremendously important in economics, finance, business, etc., and that's just my little neck of the woods.

So-called causal modeling is of course also predictive (so more accurate terms would be non-causal predictive modeling and causal predictive modeling), but the questions are very different: "What will likely happen if a certain treatment (or intervention, or policy -- call it what you want) is applied"? Important examples again abound.

Credible non-causal predictive modeling is much easier to obtain than credible causal predictive modeling. (See my earlier related post.) That's why I usually stay non-causal, even if causal holds the potential for deeper science. I'd rather tackle simpler problems that I can actually solve, in my lifetime.

The existence of competing ferocious causal predictive modeling tribes, even just within econometrics, testifies to the unresolved difficulties of causal analysis. As I see it, the key issue in causal econometrics is what one might call instrument-generating mechanisms.

One tribe at one end of the spectrum, call it the "Deep Structural Modelers," relies almost completely on an economic theory to generate instruments. But will fashionable theory ten years hence resemble fashionable theory today, and generate the same instruments?

Another tribe at the other end of the spectrum, call it the "Natural Experimenters," relies little on theory, but rather on natural experiments, among other things, to generate instruments. But are the instruments so-generated truly exogenous and strong? And what if there's no relevant natural experiment available?

A variety of other instrument-generating mechanisms lie interior, but they're equally fragile.

Of course the above sermon may simply be naive drivel from a non-causal modeler lost in causal territory. We'll see. In any event I need more education (who doesn't?), and I have some causal reading plans for the summer:

Re-read Pearl, and read Heckman's critique.

Read White on settable systems and testing conditional independence.

Read Angrist-Pischke. (No, I haven't read it. It's been sitting on the shelf next to me for years, but the osmosis thing just doesn't seem to work.)

Read Wolpin, and Rust's review.

Read Dawid 2007 and Dawid 2014.

Last and hardly least, get and read Imbens-Rubin (not yet available but likely a blockbuster).

6 comments:

  1. Interesting. I collected a range of comments on a similar topic a while back. A little disorganized, but it has links to the originals. Also some entertaining rants. For my part, I like empirical work with loose structure as a source of facts, but it's hard (impossible?) not to have some structure if you want to make progress.

    Cheers.

    Link: http://pages.stern.nyu.edu/~dbackus/Identification/ms/Notes_Friedman_on_ident_Dec_13.pdf

    ReplyDelete
    Replies
    1. Very interesting notes. I've now updated the post to include Rust's review of Wolpin on the "summer reading list."

      Delete
  2. I think that predictive modeling can play a large role in causal inference. There are two big issues in causal inference: 1) who gets treated? 2) what would have happened if they weren't treated? The first problem involves predicting who is treated, the second problem is predicting the counterfactual. Better models for prediction lead to better models for causal inference.

    ReplyDelete
  3. Phil Dawid's notes "Fundamentals of Statistical Causality" are also worth adding to this list:

    http://www.ucl.ac.uk/statistics/research/pdfs/rr279.pdf

    My understanding is that these contain most of the material he presented at Wharton this Spring.

    ReplyDelete
    Replies
    1. Yes of course! How could I forget. I'll add it to the post. Thanks.

      Delete
    2. A short introduction to the longer set of notes I mentioned above just popped up on ArXiv:

      http://arxiv.org/abs/1405.2292

      Delete