Magazine | April 6, 2015, Issue

Modern Pollster

(Blend Images/Hill Street Studios)
The profession must adapt to cell phones and the Internet

A few days before Eric Cantor lost his Virginia congressional district’s GOP nomination last year, his campaign touted the finding of an internal poll. It showed Cantor, the majority leader of the House, beating his challenger, Dave Brat, 62 percent to 28 percent. As good as this 34-point lead looked, its release signaled that Cantor sensed trouble. He was right to fret: On June 10, voters favored Brat by eleven points.

Cantor’s defeat marked one of the most startling upsets in recent political history, and official Washington, worried whenever an incumbent falls, wondered what went wrong. Had Cantor neglected his constituents? Had he irritated conservatives over immigration? Had late-deciding voters broken for Brat in the final hours? Whatever the factors, just about everybody agreed on one thing: The pundits didn’t see it coming.

“It was an aberration,” says John McLaughlin, the pollster who worked for Cantor. He blames the surprise on sabotage — i.e., Democrats who took advantage of Virginia’s open-primary laws to cast protest votes against a GOP heavyweight.

Whatever the causes of the blindsiding, the plain fact is that polling is getting harder, especially at the local level, and the supposed aberrations could become routine. Pollsters are scrambling to keep up with changes in technology and behavior that have rendered traditional survey methods obsolete. “We’re facing serious challenges,” says Scott Keeter of the Pew Research Center. Steve Mitchell, a longtime pollster in Michigan, is more dramatic: “I’m not sure I’ll be able to do this for more than two or three more election cycles,” he says. “We could be watching the death of polling.”

The birth of polling came in the 19th century, as newspapers tried to gauge popular sentiment by sending reporters into the streets, armed with questionnaires and tasked with collecting opinions. By the 1930s, George Gallup, Elmo Roper, and others had improved upon these primitive practices. They pioneered new approaches that relied on sampling and probability. Even so, they suffered a few remarkable failures — none more notorious than the one represented in the iconic photo of a grinning President Truman in 1948, when he held up a copy of the Chicago Tribune with its erroneous banner headline, “Dewey Defeats Truman.” As the historian Philip White points out in Whistle Stop, his recent book on the election, Gallup and Roper separately made a series of methodological mistakes. They also had become so convinced of Thomas E. Dewey’s inevitable victory that they stopped polling before the race was really over.

Yet they were smart enough to learn from their failures, and also to take advantage of the rise of the telephone, which made it possible for researchers to call just about anybody and produce almost completely random samples of the population. Their work began to look scientific, and polling entered a kind of golden era in the 1970s that mixed relatively low costs with reliable conclusions. Ever since, most political polls have gotten the right result not just most of the time, but the vast majority of the time.

Recent elections, however, have presented new challenges. In 2012, GOP presidential nominee Mitt Romney was certain that he would win. Anybody who doubts the sincerity of his conviction need only watch the opening scene of Mitt, the Netflix documentary on his campaign. “Does someone have a number for the president?” said Romney on Election Night, recognizing for the first time that he would have to concede. “Hadn’t thought about that.” The reason he hadn’t thought about it was that his pollsters had fed him lousy data — information that Romney and many of his supporters chose to believe, even as it contradicted other surveys that showed the race tipping to President Obama.

The polls of 2014 saw nothing quite so spectacular, and they did a fairly good job of picking winners and losers. Yet they commonly underestimated the strength of GOP candidates — a flaw on display perhaps most prominently in Virginia’s Senate race, in which Republican Ed Gillespie nearly ousted Mark Warner, the Democratic incumbent. Most polls had predicted a double-digit win for Warner, who in the end prevailed by less than a percentage point. Many Republicans wondered whether more-accurate polling would have boosted GOP turnout and generated a different outcome.

So what’s going on? Several factors have conspired in recent years to disrupt longstanding practices in public-opinion research, making it more difficult and expensive to arrive at reliable results. They can be summed up in two words: contact and cooperation.

The rise of cell phones has made it easier than ever to reach people, but it also has had the paradoxical effect of complicating the work of pollsters. That’s because close to half of all households have dropped their landlines, driving a wedge between phone numbers and places of residence. Pollsters no longer can look at area codes and make safe assumptions about where people live. It’s even harder among particular demographic groups, such as the young, the poor, and minorities, whose cell-phone use is even higher than average. Call-screening technologies and voicemail compound the problem by encouraging people to turn down callers they don’t know. Finally, a federal law prevents pollsters from auto-dialing cell phones — a consumer-protection measure that dates from when airtime was calculated in dollars per minute but is still on the books today. So reaching people on cell phones takes more effort, raising the costs of polling.

Even when pollsters connect with potential survey respondents, they face a new dilemma: a growing reluctance to cooperate. In 1997, according to the Pew Research Center, pollsters could count on initial conversations to turn into successful interviews more than one-third of the time. By 2012, this rate had dropped to less than 10 percent. So it takes many more calls — and deeper pockets — to yield the same results. “I used to do 300 interviews in races for the state legislature,” says Bill McInturff of Public Opinion Strategies. “Last year I was down to 250.”

A final complication for political polls involves timeliness. When pollsters conduct market research on brands and products, they can go into the field for weeks at a time, ask people what they think about Chevy and Ford or Coke and Pepsi, and emerge with trustworthy results. In politics, however, pollsters are often chasing the news and trying to spot daily trends. Clients want numbers overnight, whether they’re candidates trying to craft messages or media companies seeking a snapshot of a race following a major event. It’s even tougher when they have to burrow down to the level of legislative districts. “Pollsters are still pretty good at big public questions, such as the presidential approval rating,” says Karlyn Bowman of the American Enterprise Institute. “Local elections are getting a lot harder.”

Many pollsters are turning to the Web. The ordinary online poll, of course, is grossly unrepresentative, allowing anybody who stumbles on it to join in. Research firms, however, are striving to form large panels of survey participants whose information, when crunched the right way, provides true reflections of public opinion. “There’s a lot of trepidation because nobody knows the rules,” says Michael Link of the Nielsen Company. “The old style of polling developed over decades. Right now, we’re in the middle of a paradigm shift that’s only a few years old. We’re facing a steep learning curve.”

One experiment has shown promising results. SurveyMonkey, which helps companies, schools, and communities conduct polls of targeted audiences, regularly invites people who have finished one of its polls to take another. Using this piggybacking technique last fall, SurveyMonkey completed 135,000 interviews in the 45 states with contested races for the governorship or the U.S. Senate, keeping the results secret until after the election. Jon Cohen, a former pollster for the Washington Post, managed the project. As the results poured in, he saw something different from what the public polls were showing. “I spent October pulling out my hair, wondering why we had a Republican bias,” he says, referring to the fact that GOP candidates tended to do better in SurveyMonkey’s polls than in the polls reported by the press. Following the elections, SurveyMonkey released its results: Its polls had picked the winner in 69 of 72 races, missing only the contests for governor in Connecticut, Florida, and Maryland. Even more impressive, its predicted margins of victory were closer to the mark than those of just about everyone else, even though nobody had used a telephone.

Online polls enjoy other advantages. Over a landline, pollsters can ask people whether they think the United States is on the “right track” or the “wrong track.” On smartphones and computers, however, they can also test television advertisements, bumper stickers, campaign logos, and anything else that involves an image. John McLaughlin may have gotten burned in Eric Cantor’s primary last year, but he also helped Nathan Deal, the Republican governor of Georgia, win reelection. “We polled on the Web, so we were able to test nine ads with a panel of voters, who ranked them in order of preference,” says McLaughlin. “This helped us see what worked and what didn’t work, and to tailor messages to specific groups.” His method combined the breadth of a poll with the detail of a focus group, all in the service of winning votes.

Candidates who appear to trail their opponents have a favorite cliché: The only poll that matters is the one on Election Day. Good candidates always have known that this is at best a half-truth — and perhaps with the rise of new forms of polling, the rest will come to appreciate it as well.

John J. Miller is the national correspondent for National Review and the director of the Dow Journalism Program at Hillsdale College. His new book is Reading Around: Journalism on Authors, Artists, and Ideas.

In This Issue


Politics & Policy

Modern Pollster

A few days before Eric Cantor lost his Virginia congressional district’s GOP nomination last year, his campaign touted the finding of an internal poll. It showed Cantor, the majority leader ...
Politics & Policy

Obama at Selma

President Barack Obama’s speech in Selma, Ala., on the 50th anniversary of the “Bloody Sunday” voting-rights march drew glowing reviews, with some commentators hailing it as the greatest speech of ...
Politics & Policy

Hillary Rules

A recent New York Times report that Hillary Clinton skirted federal-records laws as secretary of state by exclusively using a private e-mail system to conduct official business set off a ...


Politics & Policy

Is the Party Over?

Following a night of drunken revelry, Homer reports, Elpenor — one of Odysseus’s unhappy and rapidly dwindling band of brothers — climbs atop the roof of the house where they ...

Books, Arts & Manners

Politics & Policy

Not-So-Grand Bargain

Many Americans today would like to move beyond the abortion wars. Most of the time, this hopeful suggestion is made either by political liberals (who are frustrated by the pro-life ...
Politics & Policy

Elite Convergence

Conservatives thinking about class and economic inequality will be attracted to Joel Kotkin’s latest book. There’s much to like in it, especially its description of how the leading elements of ...
Politics & Policy

Funky Founder

The hottest ticket in New York right now is a musical at the Public Theater, the downtown off-Broadway venue. The star and creator is Lin-Manuel Miranda, a 35-year-old Nuyorican composer ...


The Long View


To: From: Subj: DO NOT REPLY TO THIS EMAIL Thank you for contacting customer support. We have received your message and are reviewing it. We will contact you within 72 hours ...
Politics & Policy


CLASSICS Classics pile up on your desk — like unopened mail From the old man’s bank, not blank checks For endless free interpretation, But a mounting high inheritance tax, Assessing and possessing your imagination With truth, ...
Politics & Policy


Consider the Freight Train I am flabbergasted by Kevin A. Hassett’s piece “Off the Rails” (March 9), which includes the statement “Even the worst rail systems in Europe are superior to ...
Politics & Policy

The Week

‐ All our missing voters seem to have turned up in Israel. ‐ The general election in Israel was a triumph for Benjamin Netanyahu. His opponents had overtaken him in the ...

From IT

Herewith a selection of e-mails from the low-level techs who administered, written to its primary user. August 9, 2009 Hey, Jeremy here. Okay I got up and running for ...

Most Popular


Elizabeth Warren Is Jussie Smollett

Elizabeth Warren has a moving story about being fired from a teaching job because she was pregnant, a story that perfectly complements her political narrative that she is the tribune and champion of those who have been treated unfairly by the powerful. Joe Biden has a moving — and horrifying — story about his ... Read More
PC Culture

Defiant Dave Chappelle

When Dave Chappelle’s Netflix special Sticks & Stones came out in August, the overwhelming response from critics was that it was offensive, unacceptable garbage. Inkoo Kang of Slate declared that Chappelle’s “jokes make you wince.” Garrett Martin, in the online magazine Paste, maintained that the ... Read More
Film & TV

Joker: An Honest Treatment of Madness

When I saw that the New York Times and The New Yorker had run columns berating the new Joker movie, criticizing it not simply on cinematic grounds but instead insisting that the film amounted to a clandestine defense of “whiteness” in an attempt to buttress the electoral aim of “Republicans” — this is a ... Read More

The Origins of the Transgender Movement

Editor’s Note: This article has been adapted from remarks delivered at a Heritage Foundation summit. I’ve been asked to talk about the origins of transgenderism and how it relates to children and their exploitation. But first, I would like to start with a little story. Yesterday I was wandering around ... Read More