During last year’s confirmation hearings for Neil Gorsuch, the New York Times put the excesses of data journalism on display. It tried to predict Gorsuch’s voting record on the Supreme Court based on how Senator Wayne Allard voted in Congress. I’m not joking — the model simply assigned judges the ideology of their same-party home-state senators.
Now that Brett Kavanaugh is up for confirmation, the Times has offered a more plausible model based on the political donations he gave before becoming a judge:
Based on a large database of such donations, “Judge Kavanaugh is estimated to be more conservative than 66 percent of all other current and former federal judges nominated since 1980. Using the same measure, Justice Gorsuch was estimated to be more conservative than 85 percent.”
I have no doubt that political donations correlate with judges’ voting behavior in the aggregate, but how useful is this model for a single judge? Can it reveal something about an individual that we don’t already know? Put more concretely, common sense tells us that Kavanaugh will be a right-of-center justice, but should we trust this model’s specific prediction that he will be to the left of Gorsuch? To find out, we should compare judges’ donation scores with their actual voting records upon reaching the Supreme Court. Unfortunately, no donation data are available for any justice other than Gorsuch.
The database referenced by the Times does contain an imputed donation score for every judge, including those who were eventually promoted to the Supreme Court. Those scores are derived from judges’ resume characteristics, political party, clerkships, and so on. I’ve added the Republican-appointed justices to the figure using the imputed scores, but the results are not encouraging:
Souter is the second-most conservative justice according to the imputed scores, but in real life he was well to the left of every other Republican-appointed justice. The placement of Kennedy and Scalia at the endpoints does seem plausible, but the middle is a mess, with Thomas, Alito, and Roberts in the reverse order of how most court watchers would sort them.
It appears there is little these data can tell us about Brett Kavanaugh that we don’t already know from simply observing his career. That’s not to say the model itself has no value. Where the Times goes wrong is to assume that a model useful for generalizations about judicial ideology is also useful for predictions about individual judges. It’s not.