Knowledge and Prejudice

While he uses some pretty inflammatory language in reacting to my article on the foundations of social-scientific knowledge, Mark Kleiman asks what I believe to be some very important questions.

Kleiman quotes me as saying:

We should be very skeptical of claims for the effectiveness of new, counterintuitive programs and policies, and we should be reluctant to trump the trial-and-error process of social evolution in matters of economics or social policy… We need to keep stumbling forward with trial-and-error learning as best we can.

This leads Kleiman to ask:

What is this magical “trial-and-error process” that does what scientific inquiry can’t do? On what basis are we to determine whether a given trial led to successful or unsuccessful results? Uncontrolled before-and-after analysis, with its vulnerability to regression toward the mean? And where is the mystical “social evolution” that somehow leads fit policies to survive while killing off the unfit?

I devote a lot of time to this related group of questions in the forthcoming book. The shortest answer is that social evolution does not allow us to draw rational conclusions with scientific provenance about the effectiveness of various interventions, for methodological reasons including those that Kleiman cites. Social evolution merely renders (metaphorical) judgments about packages of policy decisions as embedded in actual institutions. This process is glacial, statistical, and crude, and we live in the midst of an evolutionary stream that we don’t comprehend. But recognition of ignorance is superior to the unfounded assertion of scientific knowledge.

Kleiman then goes on to ask this:

Without any social-scientific basis at all (unless you count Gary Becker’s speculations) we managed to expand incarceration by 500 percent between 1975 and the present. Is that fact — the resultant of a complicated interplay of political, bureaucratic, and professional forces — to be accepted as evidence that mass incarceration is a good policy, and the “counter-intuitive” finding that, past a given point, expanding incarceration tends, on balance, to increase crime be ignored because it’s merely social science?

My answer is yes, it should be counted as evidence, but that it is not close to dispositive. We cannot glibly conclude that we now live in the best of all possible worlds. I devote several chapters to trying to lay out some principles for evaluating when, why and how we should consider, initiate and retrospectively evaluate reforms to our social institutions.

Kleiman’s last question is:

Should the widespread belief, implemented in policy, that only formal treatment cures substance abuse cause us to ignore the evidence to the contrary provided by both naturalistic studies and the finding of the HOPE RCT that consistent sanctions can reliably extinguish drug-using behavior even among chronic criminally-active substance abusers?

My answer to this is no, and a large fraction of the article (and the book) is devoted to making the case that exactly such randomized trials really are the gold standard for the kind of knowledge that is required to make reliable, non-obvious predictions that rationally outweigh settled practice and even common sense. The major caveat to the evaluation of this specific program (about which Kleiman is deeply expert) is whether or not the experiment has been replicated, as I also make the argument that replication is essential to drawing valid conclusions from such experiments — the principle that Arnold Kling called in a review of the article, “Don’t trust one-offs.”

Jim Manzi — Jim Manzi is CEO of Applied Predictive Technologies (APT), an applied artificial intelligence software company. Prior to founding APT, Mr. Manzi was a Vice President at Mercer Management Consulting where ...

Most Popular

Elections

Romney Is a Misfit for America

Mitt’s back. The former governor of Massachusetts and occasional native son of Michigan has a new persona: Mr. Utah. He’s going to bring Utah conservatism to the whole Republican party and to the country at large. Wholesome, efficient, industrious, faithful. “Utah has a lot to teach the politicians in ... Read More
Law & the Courts

What the Second Amendment Means Today

The horrifying school massacre in Parkland, Fla., has prompted another national debate about guns. Unfortunately, it seems that these conversations are never terribly constructive — they are too often dominated by screeching extremists on both sides of the aisle and armchair pundits who offer sweeping opinions ... Read More
U.S.

Fire the FBI Chief

American government is supposed to look and sound like George Washington. What it actually looks and sounds like is Henry Hill from Goodfellas: bad suit, hand out, intoning the eternal mantra: “F*** you, pay me.” American government mostly works by interposition, standing between us, the free people at ... Read More
Film & TV

Black Panther’s Circle of Hype

The Marvel Cinematic Universe (MCU) first infantilizes its audience, then banalizes it, and, finally, controls it through marketing. This commercial strategy, geared toward adolescents of all ages, resembles the Democratic party’s political manipulation of black Americans, targeting that audience through its ... Read More