I previously wrote about the frivolous ethics complaint against Mark Regnerus, the author of the recent study on gay parenting. Now, the journal that published the study, Social Science Research, has conducted an internal audit. The audit will be published officially in the journal’s November issue, but it’s already made its way into the hands of numerous journalists, including yours truly. I have also read an accompanying essay by the journal’s editor, James Wright.
The audit is not nearly as critical — or revelatory — as Regnerus’s opponents seem to think. It merely bolsters the basic conclusion that every fair-minded observer came to long ago: Regnerus’s research certainly has its flaws, but so do the other studies on this topic.
The audit’s main purpose is to evaluate the publication process — the auditor was given access to all the correspondence relating to the study. Of course, the underlying issue is whether the Regnerus study is so unsound that it should not have been published.
One of the allegations against the journal is that when it chose scholars to publish responses alongside the study, it picked two people who had been involved in the study itself — a conflict of interest. Ironically, the person chosen for the audit, sociologist Darren E. Sherkat, is also not a neutral observer: He has been hurling scatological references at the study since it was published. One blog post he wrote was titled “The Gold Standard for Right Wing Propaganda” and called the study a “piece of sh[**].” The comments he’s made since the audit are in precisely the same vein.
#more#Interestingly, the audit has little negative to say about the publication process. In Sherkat’s view, a couple of the confidential peer reviewers should have been more forthright about having written papers with Regnerus more than a decade ago, though all the reviewers are respected scholars who are qualified to comment on this topic, and all recommended publication. The reviewers also should have been more critical. Wright should have looked more into the study’s funders, two conservative groups that are not well known. Sherkat also accuses Wright of speeding along publication in the hopes of creating a buzz (“Guilty as charged,” Wright says in his response).
But Sherkat also notes that Wright is not “part of a conservative conspiracy” against gay rights; in fact, the editor accepted one of Sherkat’s own papers last year “over the objections of two conservative reviewers.” All in all, Sherkat says, his “review of the editorial processing of the [Regnerus study and an accompanying one on the same topic] revealed that there were no gross violations of editorial procedures.”
However, Sherkat also takes what is arguably a detour, offering a detailed critique of the study itself. In particular, he criticizes Regnerus’s survey data, which were collected by a marketing firm and adjusted using statistical techniques that are proprietary to the firm and thus not disclosed. Sherkat goes so far as to say that “No study investigating such crucial questions using data collected in this manner should be published in a top-tier general interest social scientific journal” (his italics). However, he notes that “other peer-reviewed studies” have used these methods as well — this is a criticism of how things are done in social science, not a problem with this particular work.
Based on his analysis of the Regnerus study and the accompanying one, Sherkat says that “neither paper should have been published, in my opinion.” Among his suggestions for the editorial board are that it should become less “old, white, and male,” and that it should make more of an effort to make sure that peer reviewers don’t have connections to authors. Again, these are issues with the way things are done, not with this paper in particular.
Fair enough. Social science does need to raise its standards, and we should all hope for a day when academic journals can reject every study that lacks rigor in basic ways. But the simple fact of the matter is that the research on gay parenting from both sides leaves much to be desired, and in some ways the Regnerus study is a step in the right direction. In one sense, the journal’s internal audit calls on academia to take the next step — but in another, it chastises the journal for taking this one.