Last week, I highlighted law professor Todd Pettys’s critique of a much-ballyhooed study by political scientists Lee Epstein, Christopher M. Parker, and Jeffrey A. Segal that purports to show that in First Amendment speech cases “the votes of both liberal and conservative justices tend to reflect their preferences toward the ideological groupings of the speaker.” Epstein, Parker, and Segal have now provided their response (one page of text, plus some tables, plus a long appendix) to Pettys’s critique. I don’t find it persuasive. Some observations:
1. As I’ve made clear, I believe that the Epstein-Parker-Segal study suffers from basic flaws that render it incapable of generating any meaningful insights. Pettys’s critique, as I saw it, bolstered my skepticism in various ways. For example, I posited that “there may be factors other than speaker-favoritism or speaker-hostility” that would account for supposed disparities in a justice’s voting pattern (supposed disparities, I emphasize, that the study authors in fact don’t measure against any sound benchmark). Pettys finds that the campaign-finance cases explain most or all of the supposed disparity for Chief Justice Roberts and Justice Alito, and he observes that cases involving sexual speech (e.g., obscenity and child pornography) may well account for much of Justice Scalia’s supposed disparity. Further, by exposing bizarre assumptions that the study authors make (e.g., conservative justices would see racist speakers—even including anti-white speakers—as part of their ideological in-group) and by identifying speaker codings that are wrong or contestable, Pettys called into serious question whether the study, even on its own terms, was objective and workable.
2. The study authors maintain that “of the 36 challenges” that Pettys makes to case codings, “29 are not errors but coding disagreements of one form or another.” The authors refer the reader to an appendix (Excel version available at last link here) in which they set forth the bases for their disagreements.
I’ll leave it to Pettys to engage comprehensively with the study authors and will instead consider here only some of their positions.
a. The authors observe that Pettys doesn’t like the “issue area definitions” set forth in the Supreme Court Database and complain that “he’s not free to condemn our work for failing to meet” his preferred definitions. But Pettys is utterly transparent about his concerns about the Supreme Court Database, and it’s entirely reasonable for him to fault the authors for relying on a flawed database. That’s what “garbage in, garbage out” (a phrase Pettys doesn’t use) means. In response to the charge that they shouldn’t have included in a study of First Amendment speech cases some cases that didn’t involve First Amendment speech issues, I don’t see how it’s much of an answer for them to assert that they were just following the Supreme Court Database.
b. Even stranger, in response to Pettys’s objection that they incorrectly classified the ideological identities of speakers, the authors (in their appendix) repeatedly claim, in one fashion or another, that Pettys is “too focused on the ideological characteristics of the speaker rather than the speech.” (Emphasis added.) But the very point of their study, as they state in their abstract, is to use “in-group bias theory” to (supposedly) show that liberal justices are more “supportive of free speech when the speaker is liberal” and that conservative justices are more—indeed, much more—supportive of free speech when the speaker is conservative. Indeed, in their introduction they distinguish “in-group bias accounts,” in which it is the party’s “identity [that] would matter” from “standard ideological accounts of judicial behavior.”
Insofar as the authors now seem to be shifting the basis for the speaker coding from the speaker to the speech, they at the very least appear to be obscuring a troubling ambiguity in the in-group bias theory that they are trying to apply. To adapt one of Pettys’s criticisms: If we can’t be reasonably sure in a particular case whether a justice would look to the speaker or to the speech to determine whether the speaker is a member of an ideological in-group or out-group, we can’t confidently use the justice’s vote in that case to measure the justice’s tendency toward ideological in-group bias.
c. I pointed out in my initial post Pettys’s observation that the authors bizarrely classify David Dawson—a convicted murderer on death row who belonged to a white racist prison gang and who called himself a disciple of Satan—as a conservative speaker because they code racist communication and racist behavior as conservative. What do the authors say in defense? “Just because all conservatives don’t agree with the speech doesn’t make it more liberal.” That’s it.
As for their related coding as a conservative speaker an African-American man whose sentence had been enhanced because he selected his victim (a young white boy) on the basis of his race, here’s what the authors muster: “Hate crime laws are liberal laws, so speaking out against them is coded as conservative speech.”
Neither response does anything to address Pettys’s broader point: Are we really to believe that any conservative justice would see either of these two racist speakers as part of his own ideological in-group?
3. The authors contend that, with the corrections that they’ve seen fit to make, nothing in their results changes “in any substantively or statistically significant way.” In one sense, I agree with the authors, as I don’t think that their study, even if it were internally flawless, would yield anything that is “substantively or statistically significant.” In any event, I’m much less interested in any quibbling over whether particular case codings are, in the end, defensible than in how the existence of irreducibly arbitrary and questionable judgments helps to illustrate the folly of the entire endeavor.
In fairness to Epstein, Parker, and Segal, the broader question on which I differ with them is whether and when modern “political science,” as applied to judicial decisionmaking, might be a worthwhile field of endeavor. I’ll have more on that in a follow-up post.