Olaf Storbeck writes up a new paper on the effectiveness of a small number of elite public high schools in the United States:
In their paper, Atila Abdulkadiroglu (Duke University), Joshua Angrist and Parag Pathak (both: MIT) come to a staggering conclusion: Top-notch schools apparently do not influence the performance of pupils at all.
Storbeck ends his post on the following note:
Attending an elite school might have other benefits which are not addressed in the paper, for example reputation effects and a closely knit alumni network. Both possibly might have the consequence that children who attended an elite school get better jobs and earn more, although their actual performance does not really justify this.
If this is the case, elite schools might be beneficial from a personal point of view but not necessarily from a macroeconomic perspective. One might be tempted to question why the taxpayer should pay for such an operation that enhances private but not social welfare…
Well, we’d need evidence to suggest that taxpayers are paying more on a per pupil basis for these schools than for others. Storbeck’s post is based on the premise that high schools like Stuyvesant and Boston Latin enjoy lavish facilities. Having attended Stuyvesant, I can attest to the fact that the facility, which was built, if I recall correctly, by the Battery Park City Authority, was indeed cutting edge in the mid-1990s, but the facility has deteriorated considerably as school officials have mandated a student population that is higher than the “carrying capacity” of the building, due to a floorspace-to-student ratio that doesn’t factor in the inefficient allocation of floorspace in 1990s-era computer labs and perhaps to a political desire to meet the strong demand for places.
I’d argue that “consumer satisfaction,” understood in this context as a happier experience for students, is important. Is it worth paying a premium? Perhaps not. But what are the costs of a Stuyvesant or a Boston Latin per graduate as opposed to per pupil? There might be a negative externality associated with “creaming” gifted students from conventional public high schools. This, however, introduces a stock and flow problem: how many parents of children attending specialized high schools might have otherwise moved outside of the boundaries of New York city to send their children to public high schools elsewhere, and how many might have instead chosen to send their children to parochial and independent schools?
I should note that the findings in the paper comport roughly with my own experiences. Though Stuyvesant has a very strong reputation, I’ve often had to explain to friends and acquaintances that the school is quite large, with roughly 700 students in each class when I was there. A smaller high school uptown, Hunter College High School, was considerably smaller, and sent a higher proportion of its graduates to elite colleges and universities. The 100 most competitive students at Stuyvesant were indeed very competitive, with almost all attending selective schools, etc. The least competitive students fared less well. One of the main advantages of Stuyvesant is that it wasn’t plagued by disciplinary problems, and so teachers could devote themselves to teaching. Because Stuyvesant was part of the public school system, it was plagued by the same work rules as other public high schools. Sol Stern wrote a brilliant article on the subject for City Journal in 2003, plaguing the ordeal faced by one of the school’s most gifted math teachers:
While Stuyvesant admits only the city’s finest students, though a process that is free of subjectivity and nepotism, its hiring decisions are far less merit-driven. Like every other public school in the city, Stuyvesant is plagued by bureaucratic regulations and corrosive work rules that favor seniority and paper credentials over a teacher’s knowledge and skill in the classroom.
If Stuyvesant had more autonomy, perhaps it could more easily recruit and retain teachers who have a knack for reaching students in the bottom half of the school’s distribution — I was one of them, incidentally, and I clawed my way out with the help of a few people who continue to be my closest friends, all of whom have gone on to distinguished careers. I should stress that Stuyvesant had many excellent teachers. But it was a commonplace observation, even when I was a high school student, that teaching at Stuyvesant could be a kind of stress-free sinecure, where the industriousness of immigrant students would make any number of not-so-hot teachers look spectacular, or at least competent.
So to the cynics in my Stuyvesant class, this study isn’t exactly earth-shattering. What is true, and what is harder to measure, is that many of my peers were palpably less afraid of being ridiculed or attached at Stuyvesant than they might have been at a conventional public high school, whether or not those fears were justified. The enormous pressure to attend one of the elite public high schools, which has, I sense, calmed down to at least some degree in recent years as new magnet schools have been established, flowed from the collapse of discipline at conventional public schools, which might itself have been a reflection of the chaotic home lives of many New York city students.
Jesse Anttila-Hughes has raised questions about the study:
As the authors openly admit in the paper, the experimental design (regression discontinuity, which was begging to be used to evaluate NYC specialized high school outcomes) is inherently limited in what it can say about students who were not near the cutoff. By assumption one treats students who barely make it into the school as being more or less the same as students who barely fail to make it in, and thus “going to the specialized high school” can be considered quasi-randomly assigned. Given that (and all of the tests that are run to make sure this assumption is valid) it looks like the simple act of going to a specialized high school when you’re on the cusp is pretty nil. This may be surprising (and of course gets summarized in the Gothamist as “Stuyvesant, Bronx Science, Top Public Schools Not Worth It”) but I think becomes less so with a little unpacking.
First, it’s important to note that there’s a very big difference between being in the bottom 10% at one school versus the top 10% at another. Kids who barely make it into a specialized high school are competing and comparing themselves against the remaining 90% of students who had little difficulty getting in. Meanwhile, their comparables at other schools are at the high end of the distribution. Given the complex nature of peer effects, teacher attention, and everything else, I’d say that these populations end up having very different experiences.
Second, I’d argue that a major reason specialized schools exist is not to help marginal kids do better but to allow superstar kids to do extraordinarily well. Stuy is famously referred to as a “haven for nerds” and like many top schools succeeds by virtue of giving driven and talented kids the chance to do what they want and the resources to do so. I imagine it’d be difficult to tease out (perhaps something geographic? I know a lot of kids from my neighborhood in the Bronx who went to Bronx Science despite getting into Stuy because it was much closer…) but I strongly suspect that the causal effect of going to the school is hugely nonlinear in ability.
Lastly, I’d say that even if we were to grant that the local treatment effect identified in the RD design reasonably proxied for the school’s impact, test scores might not be the best place to look at outcomes. The specialized high schools are often touted as a means of leveling the playing field between poor (often immigrant) public school kids and rich private school ones. I suspect that if the outcomes of interest were not test scores but rather admission to elite colleges or wages in one’s mid-20s, the results would be rather different. In sum, the paper is super tightly identified, but given the populations they can plausibly claim to compare and the outcomes evaluated I’m not hugely surprised that the authors find little effect. [Emphasis added]
When Anttila-Hughes refers to “resources,” I should stress that the key resources were peer networks and mentorships. There were a number of science teachers, for example, with long experience of shepherding students through what we then called the Westinghouse (now Intel) science competition, and there were older students and alumni/ae who offered assistance that was at least as valuable.
I’m of the view that making top 5% a bit better is crucially important. This is not a view that is universally embraced.