On Tuesday I wrote a relatively short piece noting that new research is debunking the widely-held belief that there is a link between discriminatory behavior and so-called “implicit bias.” Yesterday, New York magazine published a long, detailed, and thorough essay that demolishes the idea that “implicit bias” — as measured by the extraordinarily popular Implicit Association Test (IAT) — can measure either real bias or meaningfully predict human behavior (hat tip, Roger Clegg). Some highlights:
A pile of scholarly work, some of it published in top psychology journals and most of it ignored by the media, suggests that the IAT falls far short of the quality-control standards normally expected of psychological instruments. The IAT, this research suggests, is a noisy, unreliable measure that correlates far too weakly with any real-world outcomes to be used to predict individuals’ behavior — even the test’s creators have now admitted as such. The history of the test suggests it was released to the public and excitedly publicized long before it had been fully validated in the rigorous, careful way normally demanded by the field of psychology. In fact, there’s a case to be made that Harvard shouldn’t be administering the test in its current form, in light of its shortcomings and its potential to mislead people about their own biases. There’s also a case to be made that the IAT went viral not for solid scientific reasons, but simply because it tells us such a simple, pat story about how racism works and can be fixed: that deep down, we’re all a little — or a lot — racist, and that if we measure and study this individual-level racism enough, progress toward equality will ensue.
The test is so unreliable that people can take it at different times and different days and get wildly different results — meaning that there’s a chance that the IAT is measuring nothing more than a person’s particular skill at the test:
What all these numbers mean is that there doesn’t appear to be any published evidence that the race IAT has test-retest reliability that is close to acceptable for real-world evaluation. If you take the test today, and then take it again tomorrow — or even in just a few hours — there’s a solid chance you’ll get a very different result. That’s extremely problematic given that in the wild, whether on Project Implicit or in diversity-training sessions, test-takers are administered the test once, given their results, and then told what those results say about them and their propensity to commit biased acts. (It should be said that there are still certain consistent patterns: Most white people, for example, score positively on black-white IAT, supposedly signaling the presence of anti-black implicit bias.)
In examining the history of the IAT, it’s clear that early on, the test’s architects and most enthusiastic proponents got ahead of themselves in their claims that the IAT accurately measured implicit bias, never fully grappling with the possibility that the test captures, or also captures, other stuff as well. But again: All the test itself measures is differences in reaction times, and if those reaction-times differences haven’t been proven to predict real-world behavior, it doesn’t make sense to tag someone with a high IAT score as “implicitly biased,” except in a very trivial sense of the term.
Why — with all its problems — is the test so popular? It’s a great way for progressives to virtue-signal:
For one thing, the test offers a lot to members of the public who are concerned about racism, whether they are white and concerned about their out-group biases, or nonwhite and concerned about the possibility that they have internalized bias against their own group. Taking the IAT is a way for them to feel like they are part of the solution. Now I get it — now I understand that my implicit bias is contributing to America’s race problem. This can explain the strange but common phenomenon of test-takers loudly broadcasting results which imply they are implicitly racist: It’s a way of signaling they’re serious about investigating their own complicity in a big, complicated system of oppression. There wouldn’t be anything wrong with that, of course, if the IAT were in fact providing test-takers useful information about their level of implicit bias.
The broader story told by the IAT is, at the moment, quite politically palatable and intuitively satisfying. Not only is implicit bias driving all sorts of racially unfair outcomes, that story tells us, but it’s something that we can detect and measure in ourselves, helping to raise our consciousness. “I think the reason behind adoption of implicit-bias training is simple: It is now the thing to do to demonstrate commitment to diversity and redressing inequality,” said Mitchell.
I’d encourage you to read the whole thing. It’s long, but the author says, “It would take thousands and thousands more words to fully lay out all the problems with the IAT and how it has been applied.” Once again junk science is leading the public astray.