Imagine that a U.S. president is considering his options vis-à-vis a rapidly developing Iranian nuclear-weapons program. First, a science adviser comes into the room and predicts that if the Iranians take the following quantity of fissile material and compress it into a sphere of the following size under the following conditions, then it will cause an explosion large enough to destroy a major city. Next, a historian comes into the room and predicts that if external attempts are made to thwart Iranian nuclear ambitions, then a popular uprising will sooner or later ensue and force changes in government until Iran has achieved nuclear capability.
The president would be unwise to begin debating the findings of nuclear physics with his science adviser. Conversely, the president would be unwise not to begin a debate with the historian. This would likely include having several historians present different perspectives, querying them on their logic and evidence, consulting with non-historians who might have useful perspectives, engaging in introspection about human motivations, considering prior life experience, and so on.
Next, an economist walks into the room. She predicts that if the CIA were to successfully execute an Iranian currency-counterfeiting scheme designed to create additional inflation in Iran for the next five years, then the change in Iranian employment would be approximately X. Is this more like the historian’s prediction or the physicist’s prediction?
Superficially, she might sound more like the physicist. She would use lots of empirical data, equations, and technical language. But many issues would remain outside the grasp of this analysis. How would consumer psychology in Iran be altered by the scheme, and how would this translate into overall demand changes? Would the economy respond to this problem over time by shifting resources to new sectors, and if so, what innovations would this create? How would inflation in Iran affect its foreign policy and in turn the policies of other nations, and how would these changes affect the Iranian economy? And so on, ad infinitum.
How would the economist respond if challenged like this? As far as I can see, she would respond with three kinds of evidence: (i) a priori beliefs about human nature, and conclusions that are believed to be derivable from them, (ii) analysis of historical data, and (iii) a review of the track record of prior predictions made using the predictive rule in question.
The physicist’s answer to challenges to the reliability of his prediction is simple: “Please view the following film footage of various huge explosions that resulted when independent evaluators combined the materials I described in the manner I described.” The reason the physicist need only concentrate on controlled experiments is that these are accepted as the gold-standard method for testing theories. Note that the first president faced with this kind of a briefing actually had an enormously expensive experiment conducted to test a nuclear bomb at Trinity Site, N.M.
The problem with the economist’s (iii) is that, in practice, so many things change in a macroeconomic event that it is not realistic to isolate the causal impact of any one factor. The economist’s findings are really observational data, and to call some of these macro events “natural experiments” is almost always to dress up rhetoric in analytical language.
Economists can use controlled experiments for certain narrow and important purposes, but they are not possible in macroeconomics, unless we order some randomly selected countries to, for example, print more currency, and others not. Therefore, while the economist might refer to prior periods in which some countries experienced inflationary episodes, she can’t test and validate a predictive rule in the way that the physicist can.
In the end, hard sciences produce a body of knowledge that lets us make practical predictions with tolerable reliability: An airplane of this design will fly; this vaccine will prevent smallpox; a mixture of these chemicals will explode when exposed to flame; and so on. On the important questions about which we seek predictive guidance from economics — “If we proceed on roughly our current path of projected deficits, will we do severe damage to the economy? If we spend a trillion dollars on stimulus in the face of the current economic slowdown, will we improve economic performance and raise employment significantly? If we cut the minimum wage, would employment go up?” etc. — it is possible to find highly credentialed economists who answer one way with great certainty, but it is just as easy to find equally credentialed economists who answer the opposite way with equal certainty.
N. Gregory Mankiw, a well-known professor of economics at Harvard and former chairman of President George W. Bush’s Council of Economic Advisers, is the author of some of the most influential economics textbooks in use today. In the fifth edition of Essentials of Economics (2008), Mankiw summarizes 14 findings that have achieved widespread acceptance among economists. Among them are:
Fiscal policy (e.g., tax cut and/or government expenditure increase) has a significant stimulative impact on a less than fully employed economy.
A large federal budget deficit has an adverse effect on the economy.
A minimum wage increases unemployment among young and unskilled workers.
In fact, 10 to 20 percent of practicing economists disagree with each of these assertions. More fundamental is that even if we assume them to be correct, they are too vague to really help settle policy arguments.
Consider the deliberations around how to respond to the economic crisis of 2008. The country was facing a terrifying situation, and there was a widespread belief that emergency measures of some kind were called for. The Obama administration proposed a large stimulus program, which led to an intense public debate in January and February of 2009. Setting aside for a moment ideological predispositions and value judgments, this presented a specific technical issue: What would be the effects of any given stimulus proposal on economic output and employment? This was a practical question worth trillions of dollars.
The role of government spending and deficits in a major economic downturn has been the subject of extensive academic study for decades, and many leading economists were active participants in the public discussion in early 2009. Paul Krugman and Joseph Stiglitz, both Nobel laureates, argued that the Obama administration’s proposed stimulus would be successful in improving economic performance. In fact, they both argued that it should be bigger. On the other hand, Nobel laureates James Buchanan, Edward Prescott, Vernon Smith, and Gary Becker all argued that it would fail to improve economic performance enough to justify the investment. This was not an argument about technical minutiae, but a disagreement about the basic effects of the policy.
While fierce debates can be found in frontier areas of all sciences, they are not all as if, on the night before the Apollo moon launch, numerous Nobel laureates in physics had been asserting that rockets couldn’t get as far as the moon, almost as many had been saying they could get there in theory but needed much more fuel — and some had been arguing that there was no moon. The only thing an observer could say with high confidence prior to the launch of the stimulus program was that at least several Nobel laureates in economics would be incorrect about its effects.
Debates also broke out in 2009 about the other two supposed examples of consensus that I have borrowed from Mankiw. James Buchanan argued that large projected federal deficits would have an adverse effect on the economy, while Joseph Stiglitz argued that “what really matters is not the size of the deficit but how we’re spending our money.” And Gary Becker argued that cutting the minimum wage would increase employment, while Paul Krugman asserted that cutting the minimum wage would do no such thing.
When it comes to deciding what policy actions to take, we should listen carefully to what economists and other social scientists say, but we should treat their assertions differently than we do predictions arrived at via experiments. We should subject them to useful cross-examination by specialists in other fields, reflect on how to weigh technical and non-technical opinions, ponder human motivation, and all the rest. Beyond this, we should always keep in mind the unreliability of social-science predictions, and treat the fog of uncertainty about the potential effects of our actions as fundamental when considering what to do. This is far from saying that social science is valueless, or that policymakers should not consult social scientists — indeed they should. But we should nonetheless be extremely humble about anyone’s ability to make reliable, non-obvious predictions about the results of potential policies.
I believe that recognition of this deep uncertainty should influence how we organize our political and economic institutions. In the most direct terms, it should lead us to value the freedom to experiment and discover workable arrangements through an open-ended process of trial and error. This is not a new insight, but the central theme of an Anglo-American tradition of liberty that runs from Locke and Milton through Adam Smith and on to 20th-century libertarian thinkers, preeminently Karl Popper and F. A. Hayek. In this tradition, markets, democracy, and other related institutions are seen as instruments for the discovery of practical methods for improving our material position in an uncertain environment. Ironically, we need freedom because we are ignorant.
— Jim Manzi is a senior fellow at the Manhattan Institute, the founder and chairman of an applied-artificial-intelligence software company, and author of Uncontrolled: The Surprising Payoff of Trial-and-Error for Business. This piece originally appeared in the May 14, 2012, issue of National Review.