Politics & Policy

Domestic Disputes

Bad social science and bad legal policy.

“If you keep a gun in your home,” a University of Pennsylvania press release said last week, “you dramatically increase the odds that you will die of a gunshot wound, according to research published in the June issue of the Annals of Emergency Medicine.” “Keeping guns at home is dangerous,” the researcher was quoted as saying. A New York Times story likewise reported: “People with guns in their homes are almost twice as likely to be killed by guns as people who do not keep them at home.” Another article echoed the same point; so do the websites of advocacy groups (here and here).

Frightening results for people who own guns, or who are thinking of buying them. Frightening but, it turns out, meaningless — another example of how unsound social science is being used in public-policy debates.

#ad#What the University of Pennsylvania study found was a statistical correlation: Gun ownership is correlated with gun deaths. But that two things are correlated doesn’t prove that one causes the other. The sex-crime rate is correlated over time with the use of air conditioning, but not because air conditioning causes sex crime; rather, both rise during the summer months. Likewise, whether someone in your home has been to the hospital recently is correlated with death in your home, but not because hospital care tends to kill people (though sometimes it does). Rather, both hospital stays and deaths often have a common cause: serious illness.

It turns out that a hugely disproportionate fraction of homicide victims are themselves criminals — criminals do dangerous things, and deal with dangerous people. In a recent San Francisco study, two-thirds of all gun-homicide victims (and one-third of all gun suicides) were found to have had arrest records, and other studies of gun-homicide victims yield similar results. And criminals, especially drug dealers and gang members, are particularly likely to own guns; most gun owners aren’t criminals, but many criminals are gun owners. So even if gun ownership and gun homicide are correlated, both may be caused by a common factor: Hardcore criminals are especially likely to own guns — and to be killed by guns.

Social scientists have long recognized that correlation doesn’t prove causation, because the two correlated things may actually be caused by a third thing. That’s why sound social science requires that scientists use statistical tools that control for a wide variety of such “confounding factors.”

This particular study in fact tried to control for various factors. Before controlling for those factors, the study actually found that gun-homicide victims were slightly less likely to own guns than the nonvictims were. The correlation between gun ownership and gun homicide arose when the study controlled for sex, race, age, marital status, education, income, veteran status, region, population of area of residence, and whether the person lived alone.

The study, however, completely failed to control for what might well be the most important factors: whether the household contained violent criminals, gang members, drug dealers, and the like. These are the very factors that might cause both gun ownership and gun death. And because the study didn’t control for them, it says nothing about whether gun ownership really “increases the odds” that a law-abiding citizen will be killed. The study’s results could easily flow simply from the huge set of homicide victims who are themselves criminals.

Curiously, the study starts by pointing out that an earlier study, which used a similar approach, “inspired a paper trail of challenges that continues to grow.” The new study was partly a reaction to those challenges. But a key argument in those challenges was precisely that the earlier study didn’t adequately control for the victim’s criminal background — and yet the new study made the same mistake, and in fact control for fewer factors than the earlier study did. The new study didn’t even try to deal with the possibility that law-abiding citizens aren’t endangered by their own guns, but rather that gun-owning criminals are jeopardized by their gun-owning criminal associates or enemies.

To its credit, the University of Pennsylvania study does mention this potential problem (though the press release and the New York Times article don’t). “The results could be biased,” the study says, “because risky behaviors like drug dealing could not be controlled with the data used here.” But, the study argues, this confounding “is not likely to explain the results completely” — if the victim’s own criminal background caused both the gun ownership and the victim’s death, “a gun in the home should be associated . . . with homicide by all means,” and not just guns. This, though, is unpersuasive: Drug dealers or gang members may be especially likely to kill each other precisely with guns, and not knives, clubs, or bare hands.

Likewise, the study finds a correlation between gun ownership and suicide — but doesn’t control for the obvious possibility that some people bought the guns precisely because they’re suicidal. The study points to another study that supposedly shows that gun ownership is correlated with suicide even when the gun had been owned for years (though much less strongly than when the gun has been owned for only a short time). But that other study is itself similarly flawed: It controls only for age, sex, and geography, and ignores other important factors such as history of involvement with drugs and with drug dealers, a history of chronic depression, or other personality or environmental traits.

Unfortunately, this is how conventional wisdom is molded. A badly flawed study leads to an even more flawed New York Times article. Readers read it and say “Wow, it’s dangerous for me to own a gun” — or “Since guns endanger even their owners, there’s really no reason to keep them legal.” Precisely because the study seems so authoritative, so scientific, it’s likely to be influential, even when it’s misdesigned and misreported. And this is especially so when these flaws are repeated in study after study, as they have routinely been in the gun debate. Bad social science leads to bad legal policy.

Finally, the study concludes with a recommendation to the medical profession: Physicians should “discuss with all patients” “the consequences of having access to guns.” But “discussions” are only helpful if the physician actually knows what he’s talking about. Many physicians have little personal knowledge about guns, and haven’t read the rebuttals to these studies. If they start spreading this erroneous information to their patients, the results won’t be good either for the patients or for the reputation of the medical profession.

Eugene Volokh teaches First Amendment law at UCLA School of Law.

Most Popular

Film & TV

Why We Can’t Have Wakanda

SPOILERS AHEAD Black Panther is a really good movie that lives up to the hype in just about every way. Surely someone at Marvel Studios had an early doubt, reading the script and thinking: “Wait, we’re going to have hundreds of African warriors in brightly colored tribal garb, using ancient weapons, ... Read More
Law & the Courts

Obstruction Confusions

In his Lawfare critique of one of my several columns about the purported obstruction case against President Trump, Gabriel Schoenfeld loses me — as I suspect he will lose others — when he says of himself, “I do not think I am Trump-deranged.” Gabe graciously expresses fondness for me, and the feeling is ... Read More
Politics & Policy

Students’ Anti-Gun Views

Are children innocents or are they leaders? Are teenagers fully autonomous decision-makers, or are they lumps of mental clay, still being molded by unfolding brain development? The Left seems to have a particularly hard time deciding these days. Take, for example, the high-school students from Parkland, ... Read More
PC Culture

Kill Chic

We live in a society in which gratuitous violence is the trademark of video games, movies, and popular music. Kill this, shoot that in repugnant detail becomes a race to the visual and spoken bottom. We have gone from Sam Peckinpah’s realistic portrayal of violent death to a gory ritual of metal ripping ... Read More

Romney Is a Misfit for America

Mitt’s back. The former governor of Massachusetts and occasional native son of Michigan has a new persona: Mr. Utah. He’s going to bring Utah conservatism to the whole Republican party and to the country at large. Wholesome, efficient, industrious, faithful. “Utah has a lot to teach the politicians in ... Read More
Law & the Courts

What the Second Amendment Means Today

The horrifying school massacre in Parkland, Fla., has prompted another national debate about guns. Unfortunately, it seems that these conversations are never terribly constructive — they are too often dominated by screeching extremists on both sides of the aisle and armchair pundits who offer sweeping opinions ... Read More