A Flawed Landmark
A much cited paper contains some errors, but it doesn’t matter as much as you’d think.


Carmen Reinhart and Kenneth Rogoff, of Harvard University, have become household names — or what passes for that among economists — thanks to two very timely pieces of work in the past few years. One, their 2009 book This Time Is Different, assessed the history of financial crises and documented that the recessions they induce can be much more severe than other downturns. This conclusion made it a popular citation for liberals trying to defend the pace of the Obama recovery, as Bill Clinton did in his 2012 convention speech.

The second, a study entitled “Growth in a Time of Debt,” was also remarkably convenient, but for the other side: It compared growth rates for wealthy countries over the past 70 years against their government debt loads at the time — and concluded that countries whose ratio of government debt to GDP rises above 90 percent see a dramatic slowdown in economic growth. That made it a popular piece of evidence for American conservatives trying to sell immediate budget cuts to hold the U.S. at or below that threshold, and for European financial officials calling for austerity in high-debt nations.

Until, that is, last week, when three economists at the University of Massachusetts (Thomas Herndon, Michael Ash, and Robert Pollin) published a critique of the paper, pointing out what they saw as three errors. Austerity opponents, however confident they may already have been of their economic case, rejoiced, suggesting that the case for austerity had been dismissed with a few short academic points.

They are right that there are real problems with the Reinhart-Rogoff paper — and, more important, with the way the high-debt/slow-growth link has been used by some to support austerity. But there are a couple of problems with the criticisms, too: The study’s conclusion is not totally invalidated, just weakened, and it was never as important to the motivations or justifications for austerity as has been claimed.

The criticisms were threefold. First, when performing Excel calculations, Reinhart and Rogoff left out five countries from the study (sorry, Austria, Australia, Belgium, Canada, and Denmark) by accident, literally forgetting to highlight those nations’ data. Luckily, this made no significant difference. The correction is no more than a gotcha.

Two other issues make a difference in the findings. First there is the authors’ omission of data from certain periods, such as France in the 1970s, when that country had very high levels of debt and respectable growth. As Reinhart and Rogoff explained in the Wall Street Journal last week, they excluded these periods because the data simply weren’t available at the time, and in later, related papers, such observations have been included. Still, it does diminish the strength of the relationship they found.

Last, Reinhart and Rogoff essentially counted each country as one data point (or technically four, for its growth rate at each of four tiers of debt), rather than counting each year for each country as its own data point. Greece, for instance, has spent 19 years between 1946 and 2009 with debt over 90 percent of its GDP, while New Zealand has spent just one since 1950, yet the average growth for each country during periods of high debt is weighted equally. In one sense, this is understandable, as the UMass critics admit, since Greece’s high-debt experience shouldn’t be 19 times as important as that of New Zealand. But on the other hand, one data point for New Zealand, its single year of high debt (which saw –7.6 percent growth), probably shouldn’t be considered just as important as other countries’ extensive high-debt histories.

There’s no reason to discard Reinhart and Rogoff’s work and methodology entirely, but there are problems with it. So what happens when one takes the same data, fixes the Excel error, weights all years equally, and includes the data the famous paper excluded — that is, adopts all the recommendations of the critique, which probably go too far? Short answer: The high-debt/slow-growth correlation is weaker, but does not vanish altogether.

Growth still decreases noticeably as debt increases: With debt between 0 and 30 percent of GDP, average growth is 4.2 percent; 30 to 60 percent, it’s 3.1 percent; 60 to 90 percent, 3.2 percent; and when debt is above 90 percent of GDP, growth drops to 2.2 percent. The first three numbers are quite close to Reinhart and Rogoff’s original findings, but above 90 percent, there is a big difference: The critics’ assessment finds growth of 2.2 percent, whereas the celebrated paper had found average growth of –0.1 percent.


Sign up for free NRO e-mails today:

Subscribe to National Review