NYT Shows How Not to Analyze Mass-Shooting Data

by Robert VerBruggen

Some colleagues sent me the New York Times article “What Explains U.S. Mass Shootings? International Comparisons Suggest an Answer” today. My jaw just about hit the floor when I saw the chart that appears at the top of the piece, above everything else except the title and byline:

Uh, notice a problem there, guys? You can’t compare across countries without adjusting for population. A country with more people, all else equal, will have both more guns and more mass shootings, driving a false correlation between the two.

They get to the adjusted data about halfway through the piece. It’s true that the U.S. and Yemen are the countries with the most guns and the most shooters — and that is something that could, plausibly, suggest that easy access to guns increases mass shootings. But also worth noting is that there’s no pattern visible for the entire rest of the data set. Gun ownership varies from basically nonexistent to above 30 guns per 100 people, yet these variations don’t obviously correspond to variations in mass shootings.

There are a few other issues with the piece worth pointing out, in no particular order:

Their mass-shooting numbers and much of their analysis — including a claim that gun ownership correlates with mass shootings even after the U.S. is excluded and overall homicide rates are taken into account — come from the researcher Adam Lankford, who has refused to share information about his analysis with media outlets likely to criticize him.

In one paragraph, they claim that more gun ownership means more “gun murders,” but two of the four sources they link document gun deaths including suicide.

We shouldn’t care about “gun murders” or “mass shootings”; we should care about murders in general and mass killings in general, regardless of how they’re accomplished. (Up to a point it’s essentially tautological to claim that more guns translates to more problems with guns, because a society with no guns by definition cannot have any problems with them.) As I’ve noted numerous times before, there is no simple, consistent correlation between gun ownership and murder or homicide rates in general, either among developed countries or among U.S. states. More sophisticated studies face a variety of serious methodological obstacles — I don’t find any of them that compelling — and have reached varying conclusions. The research on mass shootings in particular is in an even more primitive state.

They claim that the U.S. has more lethal crime than other developed countries but not more crime in general, and argue that guns are the difference. Even if we take the initial claim at face value, in blaming guns we encounter the problem that, as Steven Pinker once put it, “If you subtract out all the gun homicides in the United States and you just look at the homicides committed with, say ropes, candlesticks, and daggers, we still kill people at a higher rate.” I went through various reasons for our high rate of lethal violence here.

I wouldn’t purport to know for a fact that gun ownership doesn’t lead to higher homicide or mass-shooting rates, but the effects are far more complicated and subtle than gun controllers want us to believe.

The Corner

The one and only.