Google+
Close
‘Right Side of History,’ or Primed to Say Yes?
When asking about same-sex marriage, polling methods matter.


Text  


Comments
1296

It turns out that Gallup did not always prime with a question on the legality of homosexual relations before asking about same-sex marriage. Back when it varied its practice — priming on some surveys and not others — support for same-sex marriage varied. When Gallup did not prime, support for SSM totaled, on average, 6 to 7 percentage points less than when it did. A few percentage points may not seem like much, until we recall last month’s Gallup survey: Swing 6 or 7 points in the other direction and you would bring the poll to near-equilibrium between supporters and opposers. Thus a majority of Americans might not — or at least not yet — actually support same-sex marriage. The Rice study did not prime its respondents, and it asked the question differently; the results show notably greater opposition than support.

It’s impossible to know if such priming continues to affect Gallup’s numbers today, because it no longer varies its practice of priming — it now always primes — even though the wisdom of asking about the legality of “homosexual relations” makes little sense in our post–Lawrence v. Texas era. So why does Gallup still prime its survey respondents in this way? Consistency? Perhaps, but varying the practice is a methodological safety mechanism.

Advertisement
The lack of clarity about polling extends to actual voting behavior as well. In 2010 Patrick Egan, assistant professor of politics and public policy at New York University, compiled ten years of polling data about same-sex marriage in states that had voted on same-sex-marriage ballot initiatives. He found that public-opinion polls consistently underestimated ballot-box opposition to SSM. Egan noted that “the share of voters in pre-election surveys saying they will vote to ban same-sex marriage is typically seven percentage points lower than the actual vote on election day.” Why? Egan doesn’t know.

One might suspect something akin to the Bradley Effect at work in polling on this issue. Los Angeles mayor Tom Bradley, an African American, lost the 1982 California governor’s race despite being consistently ahead in the polls going into the election. Scholarly assessments of why Bradley lost focused on social-desirability bias on the sensitive issue of race. In particular, a minority of white voters was thought to have offered inaccurate polling responses for fear that, if they stated their true preference — which emerged at the ballot box — pollsters would perceive them as racially motivated.

In other words, when sensitive issues are at stake, people may feel pressure to give pollsters answers that sound enlightened, politically correct, or free of any trace of “bigotry” — a term that has reemerged as a club in the debate over same-sex marriage. Egan, however, claims that social-desirability bias is not responsible for the gap between polling and ballot-box results in this case, a gap that remained squarely in place in the May 2012 marriage and civil-union referendum in North Carolina. However, the phenomenon was mostly absent in the four ballot initiatives last November, when the pro-SSM side was likely aided by the ballot initiatives’ being attached to a presidential election.

Other suspects are the words with which survey questions are constructed. When polling organizations include the term “rights” in their question — as do Gallup, USA Today, and CNN/ORC — support for same-sex marriage is elevated: Each found 54 to 55 percent in favor. Survey respondents appear to react positively to words like “rights,” “freedom,” and “benefits,” and negatively to words like “ban.”

Recognizing this, Quinnipiac University’s pollsters stick to a very generic and brief question: “In general, do you support or oppose same-sex marriage?” The last time they asked it, in late April 2013 — about 30 days after the High Court’s twin deliberations — 45 percent of respondents reported support and 47 percent said they opposed. Eight percent were unsure.

And yet there are polls — such as the ABC News/Washington Post one last conducted in early March — in which, for no obvious reason, support for SSM runs 8 to 10 percentage points above where other polls seem to. Sampling, in the end, is a science, but a very human one.

What to conclude? First, American public opinion seems split nearly down the middle on same-sex marriage, once we account for priming, question-wording “bonuses,” and Egan’s observations of systematic underreporting of opposition. Second, the bad news for those who oppose legal recognition of same-sex marriage is that the overall, decades-long trend lines do not favor them, individual surveys aside. Battering one’s opposition with catchy memes and claims about right and wrong sides of history may be annoying, but it has been effective. Finally, many minds have not been made up. In the 2011 population-based New Family Structures Study survey, respondents were offered an “unsure” option when asked whether they agreed or disagreed with the statement “It should be legal for gays and lesbians to marry in America.” Almost one in four 18-to-39-year-olds took it. If nothing else, the Rice study reveals that such fence-sitters can move in either direction.

— Mark Regnerus is an associate professor of sociology at the University of Texas at Austin and a senior fellow of the Austin Institute.



Text  


Sign up for free NRO e-mails today:

NRO Polls on LockerDome

Subscribe to National Review