Politics & Policy

How Biased Are Polls?

It can be a matter of timing, and how questions are asked.

It has become a familiar pattern this election cycle: A poll shows President Obama several points ahead of Mitt Romney, conservatives look at the poll’s sample and discover that a significantly higher percentage of those polled are Democrats than Republicans, and, finally, the poll is dismissed as biased.

But here is another possibility: At this particular point in the race, a higher percentage of voters may be identifying themselves as Democrats. That doesn’t mean they will still see themselves as Democrats on Election Day, but for whatever reason (such as excitement over the convention) they do now. Exit polls, after all, are taken only on Election Day, and they don’t register the shifts in party ID that may have occurred in the weeks before (when some people are already sending in absentee ballots or participating in early voting).

#ad#Lydia Saad, a senior editor at Gallup, argues that if a pollster tries to control what percentage of those polled belong to the respective parties, the pollster might well be predetermining the results. “In a wave election, let’s say,” Saad explains, “there is a huge shift of voters toward a certain candidate. In that case, you’re going to see that party’s ID go up in the polls. Does that mean you push it back down? Well, no, because you might as well just decide what percentage of the vote each candidate is going to get and don’t even bother polling.”

In August, the Pew Research Center, facing criticism for a poll where the sample was skewed Democratic, made a similar argument: “Party identification is one of the aspects of public opinion that our surveys are trying to measure, not something that we know ahead of time like the share of adults who are African American, female, or who live in the South.”

Pollster Scott Rasmussen concurs. “It does not mean it’s a bad poll,” he says, “if it has an oversampling of Democrats or an oversampling of Republicans or something else that conflicts with a perception that a person might have, but it is something that you should take into consideration in evaluating, especially two polls from the same company. If a company comes out with a poll one day that shows the president leading by five points, and the next day it has a poll that shows the race tied, and you look and see the only thing that changed was that the partisan sample changed, that’s probably just noise as opposed to real significance.”

Republican pollster John McLaughlin says that if one of his survey samples had skewed heavily Democratic, he would look carefully at the other demographics: “If I got a survey back nationally that said plus ten Democratic, I would certainly be checking all my other, what I call my quality-control checks for demographics, to find out if there’s a skew in the poll. So if I had plus ten Democratic, it might be that I have 20 percent African-Americans,” instead of 12 percent.

After all, there is no precedent in recent history for an electorate that has Democrats exceeding Republicans by a double-digit margin on Election Day. From 1984 onwards, Democrats’ two best years were 1986 (when they had a six-point advantage) and 2008 (when they had a seven-point advantage). Otherwise, Democrats have been ahead of Republicans by at most four points. Considering that Republicans and Democrats constituted equal percentages of the electorate in 2010 (35 percent), it is reasonable to assume that the Democratic advantage in 2012 will be somewhere between non-existent and seven points.

“There aren’t a whole lot of people who switch back and forth between parties,” says Rasmussen, whose polls tend not to have more than a plus-two Democratic advantage. “Over the last 20 years, the number of people who in a presidential election have considered themselves Republicans has been between 32 percent and 37 percent. The number of people who have been Democrats has stayed between 35 and 39 percent in that time frame. Yes, there are changes, but there are not massive swings.”

What may also matter, in terms of differences between different polling organizations on the partisan breakdown, is when voters are asked and how the questions are framed. “It doesn’t take a lot of imagination to think of how questions raised during a poll might make someone squeamish to admit he or she is a Republican or a Democrat at the end of the survey,” Rasmussen remarks. “If the survey was all about those mean-spirited Republicans and their war on women, some people who are marginal Republicans might tell that pollster they’re independents.” Or if they are asked about Harry Reid and the Democrats not introducing a budget, “that also might pollute the numbers.”

Small factors can matter, too, such as whether respondents are asked which party they are registered with as opposed to which party they consider themselves to be a member of. Voters could be still technically registered as Democrats, for instance, but consider themselves to be Republicans now. Another factor is whether people are asked to choose between Democratic and Republican, or between Democratic, Republican, and independent.

But regardless of partisan breakdown, Republicans should be wary of taking any polls as completely accurate.

“Part of the reason the Democrats won in 2008 was that when it looked as if McCain was going to lose, some Republicans stayed home,” argues McLaughlin. “So if President Obama is in a dead-even race with Mitt Romney in so many swing states, if the Democrats can convince enough Republicans they’re going to lose, it could take a one-point loss for the Democrats to a one-point win.”

— Katrina Trinko is an NRO reporter.

Katrina Trinko — Katrina Trinko is a political reporter for National Review. Trinko is also a member of USA TODAY’S Board of Contributors, and her work has been published in various media outlets ...

Most Popular

Politics & Policy

Students’ Anti-Gun Views

Are children innocents or are they leaders? Are teenagers fully autonomous decision-makers, or are they lumps of mental clay, still being molded by unfolding brain development? The Left seems to have a particularly hard time deciding these days. Take, for example, the high-school students from Parkland, ... Read More
PC Culture

Kill Chic

We live in a society in which gratuitous violence is the trademark of video games, movies, and popular music. Kill this, shoot that in repugnant detail becomes a race to the visual and spoken bottom. We have gone from Sam Peckinpah’s realistic portrayal of violent death to a gory ritual of metal ripping ... Read More

Romney Is a Misfit for America

Mitt’s back. The former governor of Massachusetts and occasional native son of Michigan has a new persona: Mr. Utah. He’s going to bring Utah conservatism to the whole Republican party and to the country at large. Wholesome, efficient, industrious, faithful. “Utah has a lot to teach the politicians in ... Read More
Law & the Courts

What the Second Amendment Means Today

The horrifying school massacre in Parkland, Fla., has prompted another national debate about guns. Unfortunately, it seems that these conversations are never terribly constructive — they are too often dominated by screeching extremists on both sides of the aisle and armchair pundits who offer sweeping opinions ... Read More

Fire the FBI Chief

American government is supposed to look and sound like George Washington. What it actually looks and sounds like is Henry Hill from Goodfellas: bad suit, hand out, intoning the eternal mantra: “F*** you, pay me.” American government mostly works by interposition, standing between us, the free people at ... Read More
Film & TV

Black Panther’s Circle of Hype

The Marvel Cinematic Universe (MCU) first infantilizes its audience, then banalizes it, and, finally, controls it through marketing. This commercial strategy, geared toward adolescents of all ages, resembles the Democratic party’s political manipulation of black Americans, targeting that audience through its ... Read More