Politics & Policy

A Flawed Landmark

A much cited paper contains some errors, but it doesn’t matter as much as you’d think.

Carmen Reinhart and Kenneth Rogoff, of Harvard University, have become household names — or what passes for that among economists — thanks to two very timely pieces of work in the past few years. One, their 2009 book This Time Is Different, assessed the history of financial crises and documented that the recessions they induce can be much more severe than other downturns. This conclusion made it a popular citation for liberals trying to defend the pace of the Obama recovery, as Bill Clinton did in his 2012 convention speech.

The second, a study entitled “Growth in a Time of Debt,” was also remarkably convenient, but for the other side: It compared growth rates for wealthy countries over the past 70 years against their government debt loads at the time — and concluded that countries whose ratio of government debt to GDP rises above 90 percent see a dramatic slowdown in economic growth. That made it a popular piece of evidence for American conservatives trying to sell immediate budget cuts to hold the U.S. at or below that threshold, and for European financial officials calling for austerity in high-debt nations.

#ad#Until, that is, last week, when three economists at the University of Massachusetts (Thomas Herndon, Michael Ash, and Robert Pollin) published a critique of the paper, pointing out what they saw as three errors. Austerity opponents, however confident they may already have been of their economic case, rejoiced, suggesting that the case for austerity had been dismissed with a few short academic points.

They are right that there are real problems with the Reinhart-Rogoff paper — and, more important, with the way the high-debt/slow-growth link has been used by some to support austerity. But there are a couple of problems with the criticisms, too: The study’s conclusion is not totally invalidated, just weakened, and it was never as important to the motivations or justifications for austerity as has been claimed.

The criticisms were threefold. First, when performing Excel calculations, Reinhart and Rogoff left out five countries from the study (sorry, Austria, Australia, Belgium, Canada, and Denmark) by accident, literally forgetting to highlight those nations’ data. Luckily, this made no significant difference. The correction is no more than a gotcha.

Two other issues make a difference in the findings. First there is the authors’ omission of data from certain periods, such as France in the 1970s, when that country had very high levels of debt and respectable growth. As Reinhart and Rogoff explained in the Wall Street Journal last week, they excluded these periods because the data simply weren’t available at the time, and in later, related papers, such observations have been included. Still, it does diminish the strength of the relationship they found.

Last, Reinhart and Rogoff essentially counted each country as one data point (or technically four, for its growth rate at each of four tiers of debt), rather than counting each year for each country as its own data point. Greece, for instance, has spent 19 years between 1946 and 2009 with debt over 90 percent of its GDP, while New Zealand has spent just one since 1950, yet the average growth for each country during periods of high debt is weighted equally. In one sense, this is understandable, as the UMass critics admit, since Greece’s high-debt experience shouldn’t be 19 times as important as that of New Zealand. But on the other hand, one data point for New Zealand, its single year of high debt (which saw –7.6 percent growth), probably shouldn’t be considered just as important as other countries’ extensive high-debt histories.

There’s no reason to discard Reinhart and Rogoff’s work and methodology entirely, but there are problems with it. So what happens when one takes the same data, fixes the Excel error, weights all years equally, and includes the data the famous paper excluded — that is, adopts all the recommendations of the critique, which probably go too far? Short answer: The high-debt/slow-growth correlation is weaker, but does not vanish altogether.

Growth still decreases noticeably as debt increases: With debt between 0 and 30 percent of GDP, average growth is 4.2 percent; 30 to 60 percent, it’s 3.1 percent; 60 to 90 percent, 3.2 percent; and when debt is above 90 percent of GDP, growth drops to 2.2 percent. The first three numbers are quite close to Reinhart and Rogoff’s original findings, but above 90 percent, there is a big difference: The critics’ assessment finds growth of 2.2 percent, whereas the celebrated paper had found average growth of –0.1 percent.


When Reinhart and Rogoff took an even more expansive view than they did their original paper, and looked at data since the 18th century, they found numbers similar to the ones that their critics found with just post–World War II data. Other studies have also yielded similar results. So yes, there is a correlation between high debt and significantly slower growth, but there isn’t a “trigger” (a non-linear correlation) that deflates economic growth when government debt reaches a certain point.

But now that you understand the controversy and what the evidence actually says, here’s the bad news: It doesn’t matter that much.

For one, it never made that much sense to cite the Reinhart-Rogoff study as evidence that debt has crippling effects on economic growth, because the paper sets out a correlation, not a causation.

As to which way the causality works, it seems more likely that slow growth causes high debt than vice versa: It drives up social-welfare spending, depresses tax revenues, and makes the denominator of the debt-to-GDP ratio grow more slowly.

#ad#Debt and deficits, of course, do present an obstacle to economic growth. Almost every economist agrees that excessive borrowing eventually slows growth and crowds out private investment in a seriously harmful way — something that will be a problem on our current fiscal trajectory. High debt also limits the ability of a central bank to ease policy in a recession, which is probably even more important than the fiscal constraints it imposes. But the relationship will never be as clear as the Reinhart-Rogoff paper made it look, and there is no compelling empirical argument that the U.S. is nearing a growth cliff because of its debt, as the paper originally was taken to suggest.

So should those who supported austerity on the basis of Reinhart and Rogoff’s correlation be embarrassed? No more than liberal economic commentators who rail against austerity purely based on basically the opposite correlation. They claim that fiscal contractions have been devastating to European economies merely because there’s a correlation between austerity and slower growth — shoving aside the possibility that the worst-off European economies have had to implement the most contractionary fiscal policies.

It’s enough, really, to force both sides to remember the capricious nature of the social sciences — something many conservatives were rather good at until Kenneth Rogoff and Carmen Reinhart wrote a paper with a very convenient conclusion.

This shouldn’t be too wrenching, since the political battle lines were never drawn by the data anyway. American conservatives who want dramatic short-term deficit reduction really just want a much smaller government; if they actually do fear a debt crisis that cripples growth, presumably they will still believe it even when one piece of their empirical case has been weakened. Liberals who oppose almost any spending reduction because they believe government spending is justly and efficiently allocated will not win the argument because they found an Excel error. Most Europeans who favor austerity do so because they object to subsidizing their suine southern neighbors or oppose greater fiscal integration of the euro zone.

That said, could the weakening of a prominent piece of evidence — even if it never should have been all that useful — shift the battle away from debt hawks? At the margins, it might matter a bit. Stanford economics professor John Taylor wrote over the weekend suggesting that it already has: “A participant in last Friday’s G20 meetings told me that the [Reinhart-Rogoff] error was a factor in the decision to omit specific deficit or debt-to-GDP targets” in the meeting’s conclusions.

But what does Taylor himself still think? “The main arguments now for controlling the growth of spending and gradually bringing the U.S. federal budget into balance overpower any one study, right or wrong.”

– Patrick Brennan is a William F. Buckley Fellow at the National Review Institute.

Patrick Brennan — Patrick Brennan is a writer and policy analyst based in Washington, D.C. He was Director of Digital Content for Marco Rubio's presidential campaign, writing op-eds, policy content, and leading the ...

Most Popular


The 24 Democrats

Every presidential primary ends with one winner and a lot of losers. Some might argue that one or two once-little-known candidates who overperform low expectations get to enjoy a form of moral victory. (Ben Carson and Rick Perry might be happy how the 2016 cycle ended, with both taking roles in Trump’s cabinet. ... Read More