A recent study has attempted to measure empirically the relationship between extended unemployment benefits and unemployment. It claims that ending the federal policy of extended unemployment benefits resulted in a large decrease in the number of unemployed.
This is an analytical result, of course, that pleases the political Right, who have seized upon the study. Progressives, naturally, have pointed out its flaws. As usual, Mike Konczal provides a clear and thoughtful analytical critique from the left. There are many, many other such critiques at varying levels of sophistication.
Konczal’s methodological criticisms are certainly correct. For clarity, this doesn’t mean that I believe that extending unemployment insurance to a couple of years doesn’t increase unemployment (I strongly suspect that it does). What it means is that the methods of the study are not sufficient to either prove the relationship or to measure its magnitude.
Konczal and other intellectually honest progressive critics of this study are at pains to distinguish its methodology from the methodologies used in studies that they prefer, but these distinctions ring pretty hollow to me. The same problems – sensitivity of results to small changes in data or assumptions, clear potential selection bias between treatment population and asserted control population, etc. – are exactly the kinds of problems that I have tried to demonstrate over and again in various widely cited studies of this type.
The problem isn’t with the analysts in the unemployment study at hand, but with the methods themselves. Regression and other related non-experimental pattern-finding methods of this type can sound hyper-technical and very gee-whiz (“support vector machines” – cool!), and they can serve various useful purposes. I have developed and deployed many such models in businesses. But they are simply not fit for the task of making reliable, non-obvious predictions for the effects of most contested policy interventions.
This seemingly nerdy issue turns out to be, in my view, a big deal. It is an important illustration of why the Hayekian critique of planning remains valid in so many areas. If we really could build regressions that would reliably predict what the impacts of various policies would be, it would be a powerful argument against certain political and economic freedoms. Why go to all the trouble of having a messy and expensive market, or states as laboratories of democracy, when we could just have a couple of professors build us a model?