I share some of Michael O’Brien’s ambivalence.  But the most important point, and the critical one for present purposes, is the first one he makes:  That wealth-based affirmative action is better in every respect—legal, moral, historical, practical, whatever—than race-based affirmative action.  That said, it is certainly vulnerable to some of the same criticisms when more talented and harder working applicants get passed over because of who their parents are or aren’t, and its use should accordingly be tempered (most obviously when it sets up the “beneficiaries” to fail, thereby doing no one any good).   
One caveat:  In some instances, taking account of an applicant’s economic background actually might not be unfair, no matter how you define fairness, because the person being admitted might actually be better qualified, in the sense that he or she is more likely to have the willingness and ability to do work at a high intellectual level.  For instance, if you have two candidates with the same SAT score, but one achieved it without private tutors and a review course, then the meritocratic decision is to admit that student, on the theory that probably he or she has more raw talent.