The Agenda

Guest Post by Arpit Gupta: Counterfactuals and Country Comparisons

One of the interesting recent trends in the social sciences is the push to think about causal effects in terms of counterfactuals. Philosophers, economists, statisticians, and so on have started to rethink causal inference. Rather than relying on correlations or rough comparisons, the rigorous way to evaluate the effect of X is to think about what would have happened if X did not take place — the counterfactual. This can open up a Pandora’s box of “what if” historical speculation, but can also help guide thinking through likely scenarios and possibilities. Though the counterfactual by definition can’t be observed, there are a number of ways to think through what could have been the case, and thinking this way can help refocus many debates. Here are a couple of examples:

1) Institutions and Growth

If you compare national institutional quality with economic growth among African countries, you find a positive relationship. That bolsters the argument of those — like Daron Acemoglu and James Robinson – who believe that institutional quality is fundamental to growth. But how do we know that the institutional quality was causally responsible for growth? It could instead be the case for instance that richer countries invest in better institutions.

One interesting recent strategy takes the tactic of comparing African ethnic groups that straddle national borders. First, they confirm that African borders frequently divide up particular tribes, driven by colonial-era border drawing:

Next, they focus on ethnicities that straddle the borders of countries with different institutional qualities. If national institutional quality is important, than members of the same ethnicity should do better in the country with better institutions, relative to the members of the same ethnicity in the country with worse institutions. In this case, looking at members of the same group in a different institutional setting provides a powerful counterfactual in thinking about institutional quality.

It turns out that ethnicities do about the same regardless of which country they are in. Instead, within ethnicity institutions seem to do a better job of predicting growth. The cross-country comparisons lead to a spurious conclusion because they didn’t fully take into account differences in ethnic mixes between countries. 

2) Medical Spending and Health

You’ve probably seen the typical studies finding that America spends much more on healthcare while having a not particularly good life expectancy. The conclusion often drawn from this is that American healthcare is unusually ineffective and needs to be reformed in one direction or another.

One problem with that conclusion is that it’s essential to account for demographic and behavioral differences in population when thinking about health. Though hospital spending drives health, there are many other factors involved — genetics, diet, smoking, exercise — and those underlying behaviors and biology differ enormously across countries. Gary Becker argues:


The best way to evaluate America’s expensive health care system would be to estimate the effects of different kinds of healthcare on the quality and quantity of health for individuals of various ages, incomes, races, and other categories. To my knowledge, no researchers have come close to doing this.


Instead, Becker mentions a paper by some UPenn researchers that looked specifically at individual health treatments:

We find that, by standards of OECD countries, the US does well in terms of screening for cancer, survival rates from cancer, survival rates after heart attacks and strokes, and medication of individuals with high levels of blood pressure or cholesterol. We consider in greater depth mortality from prostate cancer and breast cancer, diseases for which effective methods of identification and treatment have been developed and where behavioral factors do not play a dominant role. We show that the US has had significantly faster declines in mortality from these two diseases than comparison countries. We conclude that the low longevity ranking of the United States is not likely to be a result of a poorly functioning health care system.

The authors elsewhere point to US obesity and high smoking rates as one reason for a low life expectancy rate, despite a healthcare system that is, at least arguably, actually quite good and perhaps worth the higher cost.

Other evidence for this comes from a paper I mentioned on this blog before, looking at the assignment of patients to hospitals. It appears that hospital cost is often associated with worse health conditions in the service area around the hospital — perhaps because hospitals invest more in technology when dealing with sicker patients. If you just compare hospital to hospital, it looks as if higher costs don’t result in better care. But if you randomly allocate patients to the high-cost hospital, you get better results. 

Again, without a sensible counterfactual of how a patient would fare under a given hospital or country medical system, simply comparing one country to another may lead to an flawed comparison. Of course, health reform may still be sensible, and the better care of high-cost hospitals may not be financially worthwhile. But you really need to think about the counterfactuals and demography involved here to arrive at any conclusion. 

In rememberance of 100 years since Milton Friedman’s birthday, here’s another anecdote:

A Scandinavian economist once stated to Milton Friedman: “In Scandinavia we have no poverty.” Milton Friedman replied, “That’s interesting, because in America among Scandinavians, we have no poverty either.” 

Presumably, Friedman’s point here was that cross-country comparisons can’t adequately handle underlying cultural and other differences, and the higher income of virtually every ethnic group in the US relative to their home country is a powerful point of evidence in support of the robustness of American institutions. This sort of stuff crops up all the time. 


The Latest