I read that piece and scrolled through the study a bit, and there’s one issue I have that I don’t see addressed.
Basically, the researchers interviewed a bunch of people in the law profession, and put together a list of “effectiveness factors” that make a good lawyer. They studied a group of lawyers — the lawyers themselves, peers, and supervisors rated them on the effectiveness factors, and the lawyers submitted their LSAT and first-year law-school GPAs. The researchers developed tests that predicted the “effectiveness factors” better than the LSAT did. The LSAT, in fact, predicted few effectiveness factors, mainly the ones stemming from IQ/cognitive ability.
They call for more research, and they explicitly propose combining these tests with the LSAT rather than pitching the LSAT. The Inside Higher Ed piece is a little more gung-ho — it’s hard for them not to get excited about the fact that on these tests, there are few racial disparities.
I think the researchers make a convincing case for testing more skills than the LSAT alone can. But I believe they understate the importance of the LSAT, and that it needs to remain a major part of admissions: Because they only look at people who became lawyers under the current system, they run into the “restriction of range” problem addressed in The Bell Curve.
Basically, to get into law school today, you need to be really, really smart. Presuming you’re one of those elite few, other factors, like management and organization ability, will make the difference. In other words, people with super-duper-high LSAT scores aren’t systematically better lawyers than people with just super-high LSAT scores — but that doesn’t mean we should throw out the LSAT entirely, and let in people of average IQ but really good organizational skills.
On a side note, I don’t think many of the “non-cognitive” tests the researchers came up with would work when law-school admissions are actually on the line — in the experiment, subjects could answer all the questions honestly, because it didn’t matter. But if you’re being tested on “optimism,” how hard is it to say the glass is half full? In contrast, the great thing about a cognitive test is that if you get the question wrong, it means you didn’t know the answer. There’s no way to cheat without, well, cheating.