As a disciple of Jim Manzi, author of the most important book of 2012 (read it now so you can be ahead of the curve), I was struck by a recent Sasha Issenberg article in Slate on the Obama reelection campaign’s close collaboration with the Analyst Institute, a left-of-center organization devoted to using randomized controlled experiments to improve the effectiveness of campaign expenditures.
In a wide-ranging interview with Nick Schulz of The American, Jim offers a brief explanation of why experimental knowledge is so valuable:
The reason that successful sciences usually demand controlled experiments is that the world is complicated, and it’s therefore very easy to fool ourselves into confusing correlation with causality. The regression models that are the typical non-experimental basis for most of the kinds of claims you cite consistently fail to establish the asserted causal relationship between the intervention and the outcome. They observe a relationship between the intervention and the outcome (e.g., “children who attend this special class score 10 points higher on this standardized test than those who do not“), but they are not able to do what they claim to do: “control for” other factors that might also influence the outcome. At root, this is because the datasets and analytical techniques they use may seem impressive, but they are very crude as compared to the complexity of human society.
The Analyst Institute applies this logic to campaigns, as Issenberg explains. At first, experimenters tracked the impact of various get-out-the-vote (GOTV) strategies. Though this helped increase turnout among constituents considered likely to support a given candidate, it wasn’t enough to help them figure out how to move a voter from undecided to decided. This process has traditionally involved a great deal of crude guesswork, which has only become marginally more sophisticated over time. Now, however, the Analyst Institute has devised strategies to test which political messages resonate with which audiences:
Plenty of instinct and art remain in the Obama campaign’s approach to message development. The early stages of the process resemble the traditional model, with media strategists relying on massive amounts of conventional polling from outside firms to track the electorate’s mood and campaign dynamics, and on focus groups to add impressionistic texture and a venue to audition specific images and language. The ads and direct-mail brochures that emerge from this process can then be assigned to small groups of voters under experimental conditions, pitted against one another in various combinations and across different audiences.
That full testing cycle can take around two weeks. In the case of mail, that includes the time it takes to design, print, and mail a piece—and a window for polling before and after to see what impact it had on opinions. Then analysts can model the attributes of those who were moved by the mail. Is an ad about the auto bailout more likely to persuade upscale or downscale voters? Did younger voters respond differently than older ones to information about particular provisions of the health-care bill? Are attacks on Romney’s Bain record more salient with those leaning toward Obama or those leaning toward Romney?
As these methods proliferate, as they most certainly will, it will be interesting to see how they shape political discourse. My suspicion is that the “coarsening” of American politics reflects the fact that fear-based messages happened to be extremely effective, and so we’re about to live through what we might call the industrialization of fear-based politics.
It is also true, however, that the “consumers” of these messages — i.e., the electorate — will also learn and change over time, and perhaps become more resistant to the crudest kind of fear-based appeals. Or rather some “market segments” will become more resistant while others will remain susceptible. Given that the proliferation of media channels allows for more narrowcasting, we might see an interesting segmentation of political messaging. Indeed, I think we’ve actually been seeing that in the Obama campaign this cycle. The attacks on Bain Capital have been characterized as something other than attacks on the private equity industry in the elite media. “Of course job losses are a legitimate part of the free enterprise system! Hey, we’re just attacking Mitt Romney’s claim that his private equity experience made him a job creator.” This argument is not entirely baseless. But of course watching the actual campaign advertisements leaves one with a very different impression — one gets the sense that layoffs are such are profoundly wrong, and that those who engineer them can’t be trusted. The fact that people can manage this tension is hardly surprising. We all have our cognitive biases, and there are messages advanced by right-of-center politicians that I would reject in isolation. But my sense is that these tensions and contradictions are going to become even more striking and visible as a more rigorous, experiment-driven approach to campaigning matures.