I know it’s early, but my favorite article of the month comes courtesy of CNN. It’s called “math is racist,” and blames inequality in part on big data:
In a new book, “Weapons of Math Destruction,” Cathy O’Neil details all the ways that math is essentially being used for evil (my word, not hers).
From targeted advertising and insurance to education and policing, O’Neil looks at how algorithms and big data are targeting the poor, reinforcing racism and amplifying inequality.
These “WMDs,” as she calls them, have three key features: They are opaque, scalable and unfair.
Denied a job because of a personality test? Too bad — the algorithm said you wouldn’t be a good fit. Charged a higher rate for a loan? Well, people in your zip code tend to be riskier borrowers. Received a harsher prison sentence? Here’s the thing: Your friends and family have criminal records too, so you’re likely to be a repeat offender. (Spoiler: The people on the receiving end of these messages don’t actually get an explanation.)
To understand the critique, you have to understand the social justice warrior’s definition of “fair.” A fair result is one that either breaks down precisely along demographic lines (for example, if 40 percent of a city’s population is black, then 40 percent of its cops should be black) or favors preferred identity groups. In other words, if a university program admits more women than men, it’s a testament to the grit and courage of women to overcome historic discrimination. If it admits more men than women, then it’s evidence that the patriarchy is alive and well.
Corporations and other entities are constantly on the lookout for race and gender-blind methods of measuring risk, and the information era allows unprecedented access to hard numbers. To exactly no one’s surprise, these hard numbers show that risk and competence don’t break down neatly along demographic lines — that history, culture, and numerous other factors influence different people in different populations to make choices that impact their employability, insurability, or credit worthiness.
That’s not to say that all data-based models are good. If credit worthiness, for example, turns out not to be an effective predictor of, say, insurability or reliability, then the model should be abandoned. And people can certainly make bad decisions even with good information. The author, however, seems less concerned with a model’s accuracy than with its demographic impact. If a model disproportionately impacts the poor, then it’s bad. But what if the poor disproportionately engage in the very behaviors that increase risk?
So what’s the answer? Well, one proposal is to restrict information flow:
And yet O’Neil is hopeful, because people are starting to pay attention. There’s a growing community of lawyers, sociologists and statisticians committed to finding places where data is used for harm and figuring out how to fix it.
She’s optimistic that laws like HIPAA and the Americans with Disabilities Act will be modernized to cover and protect more of your personal data, that regulators like the CFPB and FTC will increase their monitoring, and that there will be standardized transparency requirements.
In other words, if the data drives “bad” decisions, then take away the data. So much for the love of science. But people are not automatons. Remove all discrimination from the world and magically erase all the historic effects of discrimination, and people will still make different choices, including choices that inflict lasting harm on themselves and their families. Employers and others who are forced to rely on the integrity and reliability of their employees or customers are entitled to sufficient information to make reasonable decisions — even if the outcome is something other than “social justice.”