Where Does High-Quality Evidence Come From?

by Reihan Salam

There was a time when children were taught that newborn infants came into the world via storks, who carried them from some otherworldly place. One assumes that this story served to sanitize the rather more involved way children are actually born. Recently, Rick Hess wrote a short critique of a paper by Bruce Baker and Kevin Welner. Baker and Welner seem to argue that cost-efficiency efforts should only be embraced after we gather high-quality evidence to suggest that these efforts will succeed. But of course we can’t know that until we pursue new approaches, whether through a decentralized trial-and-error process or through pilot programs of sufficiently large-scale.

Bruce Baker and Kevin Welner published their enthusiastic attack on the U.S. Department of Education’s Increasing Educational Productivity project in their Productivity Research, the U.S. Department of Education, and High-Quality Evidence (Full disclosure: Stretching the School Dollar, edited by Eric Osberg and yours truly, is one of the resources that ED recommends on the relevant web page). Bruce and Kevin explain that K-12 spending really hasn’t gone up as much as the numbers might suggest before getting around to explaining how textbook studies of cost-efficiency ought to be conducted. By denying that there’s much pressure on districts to pare back spending, they’re able to suggest that there’s no urgency so far as finding new efficiencies. They seem peeved that ED has not assembled gobs of such research before daring to suggest ways in which states and districts might save money or serve kids better. (I did find it curious that Bruce and Kevin never bothered to note that: a] thousands of education school researchers, including ed finance specialists, have shown almost no interest in such questions over the past half-century and b] that most cost-saving efforts in most sectors are based on sensible intuitions and experimentation rather than “rigorous science”). Their preferred standard seems to be that the feds not breathe a word about strategies for boosting efficiency or cutting costs until “peer reviewers” (e.g. the same scholars who have assiduously avoided such questions) decide the relevant research is conclusive. As best as I can tell, under the Baker-Welner standard, we would not yet be using ATMs, purchasing airline tickets online, collecting Medicare, or reading content labels on food–as each of these were adopted without the kinds of evidence that they demand. Ah, well, that’s why it’s sweet to be an academic. Ultimately, theirs is an argument that serves to justify business as usual, incidentally defending sloth and inefficiency, by setting a remarkably high bar for innovation or cost-saving strategies. It’s a triumph of academic wish lists over common sense. Kudos to ED for going with common sense. [Emphasis added]

 Someone somewhere will have to try the new approach in the absense of high-quality evidence. Storks do not carry high-quality evidence and then drop it into our laps. 

The Agenda

NRO’s domestic-policy blog, by Reihan Salam.