Tennessee’s foray into public preschool is not going well. The technical evaluation published last year by Vanderbilt University researchers Mark Lipsey and Dale Farran showed a familiar result: Preschool attenders outperformed the control group at the beginning of kindergarten, but their advantage was gone by the end of the year. In fact, the data suggested that the preschool group might have fallen behind their non-preschool peers by third grade. Nevertheless, preschool advocates pointed to an important caveat. Because the evaluation focused on a subsample of students who needed parental consent to be followed, the treatment and control groups were not fully randomized. Preschool advocates such as Timothy Bartik and Steven Barnett downplayed the study’s importance specifically for that reason.
But Professor Farran revealed some more bad news at a Brookings event last month. She now has the results from the full randomized sample, and they are roughly the same as from the subsample. If anything, the new results point more clearly toward a negative impact. In third grade, the preschool group scored significantly lower on the state math and science tests compared to the non-preschool group. (The effect sizes were –0.13 and –0.11, respectively.) The preschool group also was more likely to have behavioral problems and repeat grades, but those differences did not reach significance.
Far from helping, might preschool be hurting kids? I wouldn’t jump to conclusions from one study with small effect sizes. In my experience, governments are rarely able to have substantial lasting impacts on children one way or the other. Still, advocates often argue that the gains from preschool are so potentially large that we should move ahead with full-scale implementation even if the evidence is not airtight. I could use the Tennessee experience to turn that argument around on them: The evidence that preschool hurts kids is not definitive, but imagine the potential harm we could be doing unless we abolish the programs immediately!
Instead of rushing to implement statewide programs with uncertain impacts, we need more small-scale studies with gold-standard methodology. And funding for these studies needs to be structured in a way that compels policymakers to draw appropriate lessons. Paying for studies and then ignoring them when they don’t come out the “right” way is not helpful. We also need to investigate the specific mechanisms by which programs have positive or negative impacts. Falling back on mysteriously latent “sleeper effects” is not good enough.
I realize it is frustrating to hear the “more research!” refrain, but it really is needed in this case — because a new social program may not just be a waste of money; it might do harm as well.