Update: I have modified this post to correct an error that affected my third point and am reposting it in its entirety.
Last week I wrote an op-ed in the Los Angeles Times responding to an op-ed in that paper by Thomas J. Miles and Cass R. Sunstein of the University of Chicago. The Miles/Sunstein op-ed used statistical analysis of routine administrative-law cases to try to discern which justices are judicial activists. The primary point of my op-ed was to dispute their contention that their overall approach shed any meaningful light on judicial activism. In addition, I pointed out some basic statistical defects in their approach.
Miles and Sunstein have now posted a lengthy reply to me and other critics. Their reply is unsatisfactory.
First and most importantly, Miles and Sunstein say virtually nothing in response to my primary point that they’re wrong to view routine administrative-law cases as more revealing of judicial activism than “high-profile constitutional cases” are. They assert that “the number of such [constitutional] cases is small, and it isn’t easy to test competing hypotheses about partisanship and restraint.” But one need not resort to obscure statistical analysis to recognize which justices embrace and apply a view of the judicial role that most overrides the ability of American citizens to exercise their constitutional powers of self-governance.
Second, Miles and Sunstein dispute my (and Ilya Somin’s) criticism that their statistical analysis implicitly presumes that agency errors aren’t skewed in a liberal direction. Ilya Somin nicely criticizes their reply here.
Third, I had also offered this minor criticism:
Miles’ and Sunstein’s statistics are also skewed by the fact that they cover the period from 1989 through 2005. Justices Ginsburg and Breyer were not on the court in the early years of that period, when the court was reviewing agency decisions from Republican administrations that account for about a third of the total cases. The relative partisanship that Miles and Sunstein find in Ginsburg’s and Breyer’s votes would surely have been much higher if Miles and Sunstein had used the same set of cases for all justices.
I now believe that in my rush to meet a deadline I miswrote the last sentence. I should have written something like: “The relative level of overturning that Miles and Sunstein attribute to Ginsburg and Breyer votes would presumably have been higher if Miles and Sunstein had used the same balanced set of cases for all justices.” (I suspect that their relative partisanship would also have been higher, but that would depend whether Ginsburg and Breyer were more likely to uphold agency decisions of the Clinton administration that Miles and Sunstein rated conservative than agency decisions of Republican administrations that they rated conservative.)
I gather that Miles and Sunstein selected the period from 1989 through 2005 because it contains an even balance of years of Republican and Democratic administrations (including if one allows for the lag time between agency ruling and Supreme Court review). A rough balance is important to discern relative “restraint” or “activism” (as Miles and Sunstein misuse those terms), because a conservative justice who indulges his biases is more likely to defer to agency rulings in Republican administrations, whereas a liberal justice who indulges his biases is more likely to defer to agency rulings in Democratic administrations.
Because the Ginsburg/Breyer case sets were heavily skewed towards cases from the Clinton administration, the Miles/Sunstein statistics predictably understate the relative rate of overturning by those justices compared to the other justices.
Miles and Sunstein reply to the point I should have made, but their reply is to argue that they encounter “similar patterns” of judicial overturning—similar, that is, to their 1989-2005 numbers—when “we limit our analysis to the period after 1994.” Well, of course. All that limiting the analysis to the period after 1994 achieves is to skew the case set for all the justices by removing a big chunk of Bush 41 cases (as many as 36% of the total) from the analysis. In other words, in trying to correct one error, Miles and Sunstein have committed another, as they have upset the case balance for all the other justices—in the direction of understating the relative overturn rate of liberal justices and overstating the relative overturn rate of conservative justices.
If my point remains obscure, perhaps a rough baseball analogy will clarify it (at least for baseball fans). Let’s say that a statistical study shows that righthanded batters are much more likely to strike out than lefthanded batters, but it turns out that the data for lefthanded batters is based only on at-bats against righthanded pitchers, whereas the data for righthanded batters includes at-bats against both righthanded and lefthanded pitchers. My objection is that the data for lefthanded batters should also include at-bats against lefthanded pitchers. Miles and Sunstein instead reply by limiting the data for righthanded batters to at-bats against righthanded pitchers—and then think themselves somehow vindicated when the new results again show that righthanded batters are much more likely to strike out. The objective observer would fairly conclude that Miles and Sunstein don’t know much about baseball—or that they have an ideological agenda against righthanded batters.
(I am of course not contending that righthanded batters don’t strike out more often than lefthanded batters; I would guess that they do, largely because there are so many righthanded pitchers. My point is limited to the integrity of the statistical comparison.)