I’ve been meaning for a while to write about Harvard economist Jeffrey Frankel’s paper on the unreliability of economic forecasting. Frankel looks at data from 33 countries and finds a systematic bias toward over-optimism in official forecasts of GDP and budget balances. This week’s chart illustrates the magnitude of forecast errors (as a percentage of GDP) increasing as the time horizon increases. This upward trend suggests that over-optimism thrives when genuine uncertainty is higher — which is often the normal state of affairs for most of the government agencies involved in the budget forecasting process.
This matters, to me at least, because overly optimistic economic assumptions for estimates of economic growth lead to over-optimism in budget estimates. Frankel’s paper suggests that the average upward bias in the official forecast of the budget balance, relative to the realized balance, is “0.2 percent of GDP at the one-year horizon, 0.8 percent at the two-year horizon, and 1.5 percent at the three-year horizon.” However, the United States tends to be even more over-optimistic than other countries: “The U.S. forecasts have substantial positive biases around 3 percent of GDP at the three-year horizon (approximately equal to their actual deficit on average; in other words, on average they repeatedly forecast a disappearance of their deficits that never came).”
Unsurprisingly, optimism bias is more pronounced during boom times, or times of economic prosperity. But Frankel also found that optimism persists during busts: “Evidently official forecasters … over-estimate the permanence of the booms and the transitoriness of the busts.”
This is unfortunate since, as I have mentioned, adoption of a realistic economic-growth outlook can play an important role in achieving successful fiscal adjustments. For instance, it played some role in the unanticipated U.S. budget surplus of the 1990s. According to the International Monetary Fund, “when formulating the five-year budget plan, the Clinton Administration adopted a realistic growth outlook, projecting real GDP growth below 3 percent and gradually declining.” Soon after, the U.S. experienced a budgetary surplus from 1998 to 2001. Similarly, Frankel argues that U.S. budget forecasts made by the White House in 2001 and subsequent years is a case where “unrealistic forecasts were plausibly a major reason for the failure of the U.S. to take advantage of the opportunity to save during the 2002–07 expansion.” And of course, this trend continues today.
Errors in budget or economic forecasting are rampant in government. However, private-sector forecasters aren’t that much better. To be sure, most of them can’t predict big nasty recessions or smaller ones. Cardiff Garcia of the Financial Times noted a few months ago:
So economists who tend to predict near the consensus are, by definition, unlikely to anticipate extreme events, while those who correctly predict the occasional Black Swan tend to get everything else wrong (or most everything else).
Unfortunately, when it comes to economic forecasting, there’s really nowhere to turn, as the consensus view tends to miss even cyclical, non-Black Swan recessions.
Considering the implications of such forecasting errors — particularly how over-optimism contributes to excessive deficits and the perpetuation of political opportunism — improvements to the overall system would be important. The question is whether we could ever get better at forecasting the future. I wouldn’t count on it. That’s why it is more important to realize that it’s not the forecast per se that matters, but how these forecasts are used to achieve policy/political goals. It’s always a good idea for forecasters to remind readers that they should be highly skeptical of their forecasts.
I will leave the last word to the FT. In the piece, the author quotes Eric Falkenstein’s blog post from January 2011:
When I worked for an economics department, I quickly learned what a lame business we were in. Our stated purpose — to forecast the economy to allow people to make better decisions — was different than our actual purpose — to provide rationales for decisions already made, to serve as an excuse to have a get together. The sad thing is that a Big Lie needs many little lies, as the stated goal of forecasting accuracy could not be discussed openly and honestly, because if one did the stated purpose becomes untenable, and then the unstated purpose becomes unworkable. It’s one of those phony little kabuki dances that seems so quaint in primitive cultures, but just as common in our own. . . .
A problem in this field is that accuracy spells extinction because no one wants to listen to an honest forecaster, they don’t purport to know enough. Rather, listen to someone who can make you rich! In selling forecasts to the masses, honesty is a strictly dominated strategy.
By the way, I like this post where Matt Yglesias defends overly optimistic forecasts.