Politics & Policy

How Biased Are Polls?

It can be a matter of timing, and how questions are asked.

It has become a familiar pattern this election cycle: A poll shows President Obama several points ahead of Mitt Romney, conservatives look at the poll’s sample and discover that a significantly higher percentage of those polled are Democrats than Republicans, and, finally, the poll is dismissed as biased.

But here is another possibility: At this particular point in the race, a higher percentage of voters may be identifying themselves as Democrats. That doesn’t mean they will still see themselves as Democrats on Election Day, but for whatever reason (such as excitement over the convention) they do now. Exit polls, after all, are taken only on Election Day, and they don’t register the shifts in party ID that may have occurred in the weeks before (when some people are already sending in absentee ballots or participating in early voting).

Lydia Saad, a senior editor at Gallup, argues that if a pollster tries to control what percentage of those polled belong to the respective parties, the pollster might well be predetermining the results. “In a wave election, let’s say,” Saad explains, “there is a huge shift of voters toward a certain candidate. In that case, you’re going to see that party’s ID go up in the polls. Does that mean you push it back down? Well, no, because you might as well just decide what percentage of the vote each candidate is going to get and don’t even bother polling.”

In August, the Pew Research Center, facing criticism for a poll where the sample was skewed Democratic, made a similar argument: “Party identification is one of the aspects of public opinion that our surveys are trying to measure, not something that we know ahead of time like the share of adults who are African American, female, or who live in the South.”

Pollster Scott Rasmussen concurs. “It does not mean it’s a bad poll,” he says, “if it has an oversampling of Democrats or an oversampling of Republicans or something else that conflicts with a perception that a person might have, but it is something that you should take into consideration in evaluating, especially two polls from the same company. If a company comes out with a poll one day that shows the president leading by five points, and the next day it has a poll that shows the race tied, and you look and see the only thing that changed was that the partisan sample changed, that’s probably just noise as opposed to real significance.”

Republican pollster John McLaughlin says that if one of his survey samples had skewed heavily Democratic, he would look carefully at the other demographics: “If I got a survey back nationally that said plus ten Democratic, I would certainly be checking all my other, what I call my quality-control checks for demographics, to find out if there’s a skew in the poll. So if I had plus ten Democratic, it might be that I have 20 percent African-Americans,” instead of 12 percent.

After all, there is no precedent in recent history for an electorate that has Democrats exceeding Republicans by a double-digit margin on Election Day. From 1984 onwards, Democrats’ two best years were 1986 (when they had a six-point advantage) and 2008 (when they had a seven-point advantage). Otherwise, Democrats have been ahead of Republicans by at most four points. Considering that Republicans and Democrats constituted equal percentages of the electorate in 2010 (35 percent), it is reasonable to assume that the Democratic advantage in 2012 will be somewhere between non-existent and seven points.

“There aren’t a whole lot of people who switch back and forth between parties,” says Rasmussen, whose polls tend not to have more than a plus-two Democratic advantage. “Over the last 20 years, the number of people who in a presidential election have considered themselves Republicans has been between 32 percent and 37 percent. The number of people who have been Democrats has stayed between 35 and 39 percent in that time frame. Yes, there are changes, but there are not massive swings.”

What may also matter, in terms of differences between different polling organizations on the partisan breakdown, is when voters are asked and how the questions are framed. “It doesn’t take a lot of imagination to think of how questions raised during a poll might make someone squeamish to admit he or she is a Republican or a Democrat at the end of the survey,” Rasmussen remarks. “If the survey was all about those mean-spirited Republicans and their war on women, some people who are marginal Republicans might tell that pollster they’re independents.” Or if they are asked about Harry Reid and the Democrats not introducing a budget, “that also might pollute the numbers.”

Small factors can matter, too, such as whether respondents are asked which party they are registered with as opposed to which party they consider themselves to be a member of. Voters could be still technically registered as Democrats, for instance, but consider themselves to be Republicans now. Another factor is whether people are asked to choose between Democratic and Republican, or between Democratic, Republican, and independent.

But regardless of partisan breakdown, Republicans should be wary of taking any polls as completely accurate.

“Part of the reason the Democrats won in 2008 was that when it looked as if McCain was going to lose, some Republicans stayed home,” argues McLaughlin. “So if President Obama is in a dead-even race with Mitt Romney in so many swing states, if the Democrats can convince enough Republicans they’re going to lose, it could take a one-point loss for the Democrats to a one-point win.”

— Katrina Trinko is an NRO reporter.

Katrina TrinkoKatrina Trinko is a political reporter for National Review. Trinko is also a member of USA TODAY’S Board of Contributors, and her work has been published in various media outlets ...
Exit mobile version