A new survey gets an uncritical write-up in the New York Times:
In the coming weeks, thousands of college students will walk across a stage and proudly accept their diplomas. Many of them will be hungry.
. . .
A survey released this week by Temple University’s Hope Center for College, Community and Justice indicated that 45 percent of student respondents from over 100 institutions said they had been food insecure in the past 30 days. In New York, the nonprofit found that among City University of New York (CUNY) students, 48 percent had been food insecure in the past 30 days.
The first important thing to know is that, as the survey itself notes, only in its “extreme” variations is “food insecurity” often accompanied by “physiological sensations of hunger.” For instance, a student could be classified as “food insecure” if he said he worried whether his food would run out, “couldn’t afford to eat balanced meals,” and ate a bit less “because there wasn’t enough money for food.” (You can see all the questions on page 34, and on the next page the scoring system is explained. On page 7 there is an item-by-item breakdown of how students answered each question.)
Next: This survey had a very low response rate, meaning that the people who answered it are almost certainly not representative of the general population. The NYT simply ignores this fact, but here’s the survey’s explanation (emphasis mine):
Most students who were sent the #RealCollege survey did not answer it. Institutions sent survey invitations to an estimated 1,478,935 students and 85,837 students participated, yielding an estimated response rate of 5.8%. In this report, we exclude students who did not identify a college they attend.
We surveyed all students rather than drawing a subsample due to legal and financial restrictions. The results may be biased — overstating or understating the problem — depending on who answered and who did not. As readers ponder this issue, consider that the survey was emailed to students, and thus they had to have electronic access to respond. The incentives provided were negligible and did not include help with their challenges. Finally, the survey was framed as being about college life, not about hunger or homelessness.
Uh, about that. Here’s the email ad for the survey, which participating institutions were required to use word-for-word “to ensure consistency across institutions.” We’ll just say it carries a significant risk of skewing the sample toward poorer, struggling students, and that anyone trusted with designing a large survey should know it does:
Subject: #RealCollege: Speak out – chance to win $100!
Making it in college these days can be tough. We want to help.
Colleges and universities need to know about the lives of real students like you so that they can offer more support. After you complete the survey, you can enter a drawing to receive a $100 award.
This survey we call “#RealCollege” is all about you and your college experience. You’re getting it because you attend [COLLEGE NAME] and people there want to help you succeed.
Click here to share your story!
Everything will be kept confidential so, tell the truth. Share your challenges. Help us find solutions.
In addition, other — better — research on this issue is mixed at best on whether college students are any more “food insecure” than anyone else. Here’s how a 2017 Urban Institute brief summed up the situation:
Previous survey-based studies of college students have examined the prevalence of food insecurity on individual college campuses. These studies have reported widely ranging rates of food insecurity. For example, one study, examining students at the University of Alabama, identified 14 percent of students as food insecure, close to the national average (Gaines et al. 2014). However, a recent survey of community college students from 70 campuses estimated that 67 percent of respondents were food insecure (Goldrick-Rab, Richardson, and Hernandez 2017). [Goldrick-Rab is the lead author of the new survey as well. —RVB] As the authors of these reports have themselves noted, the findings are limited, in part because of nonrepresentative samples and very low response rates.
In the brief, the Urban researchers “fill an important gap in the literature by producing nationally representative numbers on the percentage of postsecondary students who are food insecure by using Current Population Survey (CPS) data from 2001 to 2015.” One might wonder why previous researchers hadn’t used this freely available, nationally representative data set rather than commissioning their own surveys.
Urban’s result:
Levels of food insecurity among households with students in four-year colleges and vocational education were 11.2 and 13.5 percent, respectively, in 2015—rates that are largely similar to national levels. However, households with students enrolled in two-year colleges were more likely to be food insecure in the period after the 2008 recession, with average rates of food insecurity of 21.2 percent during 2008–14. In 2015, the rate of food insecurity among households with two-year college students dropped to an estimated 13.3 percent.
By the way, there was an interesting study last month digging into some quirks in how college students respond to food-security questionnaires. Its results suggest that the questionnaires yielding the lowest estimates of food insecurity, such as the one used in the CPS, are the most accurate.
I figured all this out in about an hour (though I’ve written about the term “food insecurity” before), and I looked into it because the idea that 45 percent of college students are struggling to find enough to eat struck me as patently absurd. If only the journalists and editors guarding the gates between dubious findings and the national spotlight provided by the Paper of Record were similarly skeptical.