Edward B. Fiske has a very useful piece on Minding the Campus under that title.
They’re all sound reasons. I’d especially emphasize his third, that they don’t tell you what’s really important. It’s quite possible for a student to attend a “prestige” school where most of his or her professors pay scant attention to undergrads, so they coast along without much help from scholars. It’s equally possible for a student to attend a non-prestige school where the faculty pays a lot of attention to the students, engaging them in class, critiquing their writing, and exemplifying the ideals of education. The prevalent idea that students who go to “elite” schools get superior education is utter nonsense.