David Leonhardt, a smart and thoughtful columnist, believes that ARRA “made a big difference.”

As analyses by the Congressional Budget Office and others have shown, the bill was a major reason that the economy turned around in early 2009 and was growing by year’s end.

But was the CBO analysis an “independent check” on the initial CBO model of how ARRA would impact aggregate demand, spending, and job creation? Not according to Doug Elmendorf, the director of the CBO. Peter Suderman wrote a terrific post on this back in March:

In response to a question at a speech earlier this month, CBO director Doug Elmendorf laid out the CBO’s methodology pretty clearly, describing the his office’s frequent, legally-required stimulus reports as “repeating the same exercises we [aleady] did rather than an independent check on it.” CBO tweaks its models on the input side, he says—adjusting, for example, how much money the government has spent. But the results the CBO reports—like the job creation figures—are simply a function of the inputs it records, not real-world counts.
Following up, the questioner asks for clarification: “If the stimulus bill did not do what it was originally forecast to do, then that would not have been detected by the subsequent analysis, right?” Elmendorf’s response? “That’s right. That’s right.”

And of course the tax provisions of ARRA were a very big and important aspect of the legislation, as David Leonhardt reminded Ed Glaeser in June:

When you look at the program as a whole, the picture is not nearly as uncertain. Home buying jumped during the very period when a tax credit for home buying was in effect. The same happened with corporate investment. State spending stabilized in the middle of last year — just as states were hearing about their stimulus awards — even though state revenues were continuing to fall at the time. Consumer spending has risen faster than income growth would suggest, but about as fast as you’d expect given the combination of income growth and stimulus tax cuts. [Emphais added.]

Leonhardt was very convinced that the CBO’s models were sound:

Based on its economic models, the Congressional Budget Office recently estimated that between 1.4 million and 3.4 million workers who have jobs would be unemployed if the stimulus hadn’t been enacted. Three of the best-known private economic research firms — IHS Global Insight, Macroeconomic Advisers and Moody’s Economy.com — have come up with similar estimates. The average estimated effect on employment is about 2.5 million jobs.

All of these models were, of course, very similar. They made certain assumptions about how the tax credits, etc., would translate into AD. They were not empirical surveys updated with new evidence over time.

Fortunately, three researchers at the University of Michigan, Matthew Shapiro, Claudia Sahm, and Joel Slemrod, conducted just such an empirical survey, and Arpit Gupta summarized their findings at Economics 21:

More surprising is the analysis of the tax cuts associated with the 2009 ARRA stimulus. These took effect through the form of reduced federal tax withholding on payrolls, and were widely expected to be more effective than one-time tax cuts in elevating aggregate spending. Yet the authors find that a mere 13% of households reported higher levels of spending due to the tax cuts. While the deteriorating economy may account for part of the decrease in the number of households responding to the stimulus, the authors are able to isolate the change in payment mechanism as the primary reason that the 2009 stimulus was less effective than the 2008 stimulus.

The 2008 stimulus was, the authors note, extremely ineffective. One gets the distinct impression that the CBO overestimated the consumption response to the tax credits.

So it could be that David Leonhardt’s confidence in the soundness of the CBO model, and the structurally similar private forecasts, has been misplaced.

Like Alice Rivlin, I believe that ARRA was very poorly designed. And my understanding of the evidence suggests that a cautious approach to fiscal consolidation, with an emphasis on improving the effectiveness of public spending, is a much shrewder approach to an uncertain economic environment than doubling down on fiscal stimulus. But I’m certainly open-minded on this question, and I intend to take new empirical evidence into account.