A while back, I came down pretty hard on Peter Salins’s defense of the SAT. He’s written another one — this one has a lot more detail, but I find his methods confusing.

First of all, here’s basically where both of us come down. I buy Charles Murray’s argument that the SAT II is better than the SAT at predicting college grades. I buy the academic consensus that high-school GPA is better than the SAT as well. Right now colleges use a combination of SAT and GPA, and some are dropping the SAT without replacing it. I believe they’re right to drop the SAT, but they should replace it with the SAT II. Salins has been defending the SAT, with little mention of the SAT II.

Salins is a former SUNY provost and depends heavily on data he was privy to in that position. For example, take the aforementioned academic consensus that high-school GPAs do a better job of predicting college success than does the SAT. He disagrees, and here’s how he comes to that conclusion:

Nine campuses chose to increase their selectivity after 1997. … The move to raise selectivity standards had a much greater impact on entering students’ SAT scores than on their GPAs. For the rising selectivity campuses, SAT score increases between 1997 and 2001 ranged from 4.5 percent (Cortland) to 13.3 percent (Old Westbury), while high school GPAs increased only between 2.4 and 3.7 percent, a gain almost identical to that at campuses that chose not to raise their SAT standards.
The percentage increases in six year graduation rates at the rising selectivity campuses – just over a four year period — were dramatic, ranging between 10 percent (at Stony Brook whose graduation rate went from 53.8 to 59.2) to 95 percent (at Old Westbury which went from 13.3 to 35.9). Most revealingly, the seven SUNY campuses that stuck with their prior selectivity profiles, meaning their entering students’ SAT scores between 1997 and 2001 were stable or rose only modestly, actually saw their graduation rates decline.

One potential problem I see here lies in comparing a “4.5 to 13.3 percent” increase in SAT scores to a “2.4 to 3.7 percent” increase in GPA. When comparing different statistical measures like this, I believe you would use standard deviations, not percentages. For example, the difference between a 3.0 and a 4.0 student might be called “33 percent,” yet it might correspond to a 1000 versus a 1500 SAT score (a “50 percent” increase). I just made those numbers up for illustrative purposes.

(Note to statistician readers: Am I wrong about this somehow? Am I misunderstanding what he did? By a “4.5 percent” increase in “SAT scores,” does he mean that the average shifted 4.5 percent in the distribution — that is, that before, the average kid was at the 65th percentile nationwide, but now he’s at the 69.5th percentile? If you know, please e-mail comments to phibetacons[at]nationalreview[dot]com.)

The other problem I see is that, even taking his methods at face value, Salins has simply proved that if you attract students with the same GPAs, yet higher SAT scores, you’ll get higher-achieving kids. That doesn’t mean that the SAT is actually better than GPA, because he didn’t contrast it to a situation where schools sought out higher GPAs but the same SAT scores. It just shows that the SAT provides some information that GPA doesn’t. That’s a perfectly fine argument for keeping the SAT instead of scrapping it entirely (which, in fairness, is the policy he takes issue with), but it’s not an argument for keeping the SAT instead of switching to the SAT II.