Interpreting the Important New Research on D.C.’s IMPACT System

by Frederick M. Hess

So, it turns out that Michelle Rhee knew what she was doing. Stanford’s Tom Dee and the University of Virginia’s Jim Wyckoff have just published an important study on Washington D.C.’s controversial teacher-evaluation system. They find that the IMPACT system launched by former chancellor Michelle Rhee appears to boost teacher effectiveness and also makes it more likely that low-performing teachers will depart. Deservedly, he study got a lot of attention yesterday, including in the New York Times  and the Washington Post.

Making clever use of regression-discontinuity design, the authors note that IMPACT appears to aid students both by “avoiding the career-long retention of the lowest-performing teachers and through broad increases in teacher performance.”

This is good news for advocates of more sensible teacher evaluation and pay. That said, there’s also the risk that reformers will misinterpret the findings or fail to understand why the D.C. IMPACT system seems to have delivered where seemingly similar efforts have previously disappointed. Before pointing to Dee-Wyckoff as proof that aggressive teacher evaluation “works,” would-be imitators need to take a good look at what the D.C. public schools have actually done. Three key ones deserve note, especially for policymakers.

IMPACT is a program, not a statute: This means that DCPS has been able to readily and repeatedly tweak the system, year over year (and even sometimes during the year). The evaluation model is not set in stone and needn’t apply to a slew of districts with various needs; it is designed for DCPS. Efforts to legislate statewide teacher-evaluation systems, of the kind championed by the Obama administration in Race to the Top and as a condition for No Child Left Behind waivers, may be a whole different kettle of fish

Talent and Technology: Over time, DCPS has recruited and assembled a remarkable, large, and sustained team to design and implement IMPACT. They became a magnet for talent on this precisely because they were a national leader. They’ve tapped top-shelf advisers, worked assiduously to address educator concerns, and taken pains to explain the system clearly and accessibly to educators. This has been a big expense, in a district of just 45,000. It also turns out that many of the data systems in place in DCPS were inadequate for IMPACT’s needs. The district essentially had to build a parallel personnel data system in order to handle IMPACT. Indeed, many big school districts may lack even the basic information technology needed to make an IMPACT-style system work.

DCPS did IMPACT seriously: Everything about IMPACT is ambitious. The dollars for exceptional performance are huge, relative to what most districts have dabbled with. The consequences for persistent low performance are substantial. Past experience gives reason to doubt that much milder versions of the system will deliver the same jolt. Meanwhile, the program design is unusually sensible and coherent. As Dee explained, ““D.C. is fielding incentives that are just very different from what we’ve seen before. Part of that is, it’s not just cash for test scores. It’s instead incentivizing things that teachers can control more directly.”

The results on IMPACT are heartening. But it would be a mistake to allow reassuring results to serve as a justification for half-baked efforts or one-size-fits-all teacher-evaluation statutes.  Doing so runs the risk of encouraging inept efforts to scale a promising possibility.

The Corner

The one and only.