Teacher quality affects student achievement. No one doubts that. But the process of identifying, recruiting, and retaining high-quality teachers is fraught with difficulty. Simply raising pay or recruiting young Ivy Leaguers seems to have little effect on student test scores.
Now a new report from the Institute of Education Sciences (IES), a research division of the Education Department, has poured more cold water on the idea that pursuing better teachers automatically translates to better student performance. The Chicago public-school system implemented a “whole-school” intervention called the Teacher Advancement Program (TAP), which was supposed to raise the quality of its teacher workforce. Teachers were given the opportunity to earn performance pay and promotions through a special system of mentoring and observation.
Although the literature on TAP is extensive, IES focused on the most reliable study — an experimental evaluation conducted by Mathematica a few years ago. There are questions about how faithfully TAP was implemented in Chicago, and the control group was small, but here is the study’s bottom line: There were no significant gains in student test scores, regardless of grade level or subject area.
Research on teacher quality is valuable for education policy, but equally important is recognizing the limitations of intervention. We don’t know how to reliably and substantially raise academic achievement, despite the protests of “If only we just did [insert educational fad] . . .” from people unfamiliar with the literature.
But will the IES report change the conversation on teacher quality and intervention? Unfortunately, conducting evaluations and then ignoring the results is depressingly common. For example, Republicans and Democrats came together in 1998 to authorize a gold-standard evaluation of Head Start. When the evaluation showed no lasting gains, Republicans and Democrats came together again — to continue funding Head Start as if the evaluation never happened.
Head Start supporters explained away the results by arguing, among other things, that (1) not every location implemented Head Start in a “rigorous” way, and (2) lots of non-experimental research shows positive impacts. Now, in response to the IES report, the national organization that oversees TAP has made the same arguments in defense of its own program. To a cynic, this sounds suspiciously like trying to avoid the possibility of ever falsifying the claim that school interventions work. No matter the results of any evaluation, some advocates will always have a faith-based fallback position.