When we compare educational outcomes across different states and cities, it is important to keep in mind that some states and cities have populations that are (say) more disadvantaged than others. To compare the raw test scores of a public elementary school that serves affluent children raised in stable two-parent families to one that serves poor children raised in more chaotic homes would be misleading, for obvious reasons: in almost every respect, the average student at the former school come to class better prepared to learn than the average student at the latter school. We can’t identify all of the factors that could possibly influence student achievement, and so it is impossible for us to perfectly isolate the extent to which schools are “adding value” through instruction.
What we can do, however, is identify various student-level factors that are associated with achievement to help give us a clearer picture of how well our schools are performing. Matthew Chingos has done just that in a new report from the Urban Institute. That is, he has taken data from the National Assessment of Educational Progress, the yardstick that the federal government uses to measure educational performance across jurisdictions, and he has tried to make apples-to-apples comparisons by adjusting for factors like race, the language spoken at home, and socioeconomic status, among many other things, like (my favorite) the number of books in the home. “If states matter little for student achievement,” Chingos writes, “then the variation in average NAEP scores across states would be greatly diminished by accounting for the rich set of student characteristics available in these data.” But in fact correcting for student demographics only accounts for one third of the variation in educational performance.
So which states look much better when you make apples-to-apples comparisons, and which much look worse? David Leonhardt of The Upshot has written a detailed write-up of the results, and he was particularly struck by how well Texas and Florida fare when you adjust for demographics. Also interesting is the fact that in a number of states, the demographic mix makes the schools look better than they really are, e.g., Connecticut, Nebraska, Wyoming, Montana and Iowa.
Chingos’s findings bring to mind Rick Hess’s critique of “achievement-gap mania.” Yes, it is a very good thing that schools in states like Texas and Florida seem to be doing a much better job of educating children from disadvantaged backgrounds than, say, schools in California. Yet we should also be concerned about the fact that schools in Connecticut and Iowa, and many other places besides, appear to be shortchanging students from advantaged backgrounds. Let’s cheer on efforts to raise student achievement among poor kids, by all means. But we shouldn’t be satisfied with stagnant achievement among better-off kids either. To that end, I’d argue that we should be less focused on ensuring that schools are diverse (i.e., that they contain students of many different ability levels) and more on ensuring that we have a diversity of schools (so that different kinds of students get the different kinds of instruction they need). KIPP schools designed to serve poor children aren’t an ideal fit for students with affluent, educated parents who can provide a great deal of intellectual enrichment at home, and gifted and talented programs aren’t an ideal fit for students who need a great deal of help to master the basics.