Google+
Close
‘Right Side of History,’ or Primed to Say Yes?
When asking about same-sex marriage, polling methods matter.


Text  


Comments
1296

Reports of recent nationwide polls about same-sex marriage would seem to put those who oppose the idea squarely in the minority, with many who formerly opposed it apparently fearing that they would find themselves socially on the “wrong side of history,” akin to Bull Connor and his Birmingham police force or the spectators who jeered at James Meredith as he walked to class at Ole Miss in 1962.

Emotional slogans are no doubt effective, but they muddy social-scientific attempts to figure out just how popular the idea of same-sex marriage (SSM) really is in the American mind. Polling data certainly suggest that public support for SSM is increasing, and I affirm that that perception is accurate. But discerning exactly what people think about SSM — and how many of them think that way — is not as simple as a sound bite.

Advertisement
A recently released Rice University study on attitudes about same-sex marriage — and the absence of media attention accorded it — made me wonder about the science, and possible politics, behind the most commonly cited polls.

Rice sociologists Michael Emerson and Laura Essenburg analyzed data from a poll that asked a random sample of nearly 1,300 American adults — on two different occasions, in 2006 and 2012 — whether they agreed or disagreed with this statement: “The only legal marriage should be between one man and one woman.” What’s the advantage to querying the same people six years later? Rather than simply mapping trends in the overall population — which is what most polls on the subject do — you can discern internal movement within people. That is, they change their minds, and the results show that they don’t always move in the direction of greater openness to same-sex marriage.

Here is what the Rice study’s authors say they discovered: First, they found less support for same-sex marriage than polls like Gallup and CNN tend to find. In fact, in 2012, 53 percent of those surveyed agreed that the only legal marriage should be between a man and a woman, while 13 percent sat on the fence, and 33 percent disagreed with the statement. Second, they detected no statistically significant change in overall sentiment on same-sex marriage over those six years. Third, some things did change — minds — and not all of them toward favoring same-sex marriage. The authors write:

. . . when we look behind the overall numbers, we find that many people did indeed change their minds over the 6-year period. The most stable category was among Americans who agreed in 2006 that the only legal marriage should be between one man and one woman. About three-quarters (74%) who agreed with the statement in 2006 also agreed with it in 2012. Among those who disagreed with the statement in 2006, 61% also disagreed in 2012. What is surprising in light of other polls and the dominant media reports that Americans are moving in droves from defining marriage as one man and one woman to an expanded definition is the movement of people in the other direction as well, a fact missed by surveys that do not follow the same people over time.

The uncommon results of this study, when contrasted with most media reports on the matter, may be to blame for the silence observed about this release. It simply didn’t jibe with the dominant narrative of majority — and growing — support for same-sex marriage. It’s possible that the Rice study’s sample is more religious than average, given that religion was one of several topics the investigators were most interested in. (Topical interest, however, need not bias a sample if the survey contacts are conducted smartly.) Moreover, its youngest respondents were 18 in 2006 and 24 in 2012, so this survey misses out — just a bit — on the youngest adults in 2012, more of whom are no doubt on board with the shift in marital meaning. Yet what about the psychology of giving positive versus negative responses to surveys? The Rice survey is unique in that the positive response is one of support for traditional marriage rather than for same-sex marriage.

What do other surveys show, and how do they ask their questions? Gallup, the granddaddy of such organizations, regularly asks Americans, “Do you think marriages between same-sex couples should or should not be recognized by the law as valid, with the same rights as traditional marriages?” When Gallup did so just last month, 54 percent of those polled said “should” and 43 percent said “should not,” while 3 percent remained unsure. The results may be skewed by the fact that the negative response is the one favoring traditional marriage. Nevertheless, it would seem that SSM has solid support.

But Gallup continues to ask a question about the legality of “homosexual relations” before it asks about same-sex marriage, a technique known as “priming,” or preparing survey-takers for subsequent questions. In their book News That Matters, political psychologists Donald Kinder and Shanto Iyengar document how priming shapes respondents’ answers to subsequent questions, particularly where sentiments about a previous question spill over. Gallup asks whether respondents “think gay or lesbian relations between consenting adults should or should not be legal,” a question that most observers would assume is not even asked any more.



Text