The Folly of the New Online Gatekeepers

George Orwell circa 1940 (Cassowary Colorizations/Wikimedia Commons)
A progressive journalist thinks we can censor our way out of the problem of the far Right.

In March 1940, George Orwell found himself face-to-face with Fascism on the cusp of victory. Nazi Germany had subjugated the European continent, and Great Britain was preparing to resist an impending invasion that would decide the fate of the world. Orwell sought to make sense of this catastrophe by reviewing a new edition of Mein Kampf.

How had Fascism leapt from the shadows to bewitch millions? Orwell didn’t talk about the political structure of the Weimar Republic or the impact of propaganda. He talked about human psychology. Liberal modernity assumed that people needed only material comfort to be happy, but this was not true, he argued, and Fascism had stepped into the breach to offer the heroism and purpose that people, especially young male people, crave. “Socialism, and even capitalism in a more grudging way, have said to people ‘I offer you a good time,’” Orwell wrote. “Hitler has said to them ‘I offer you struggle, danger and death,’ and as a result a whole nation flings itself at his feet.” This is the kind of thing you write if you’re searching for the root cause of Fascism’s appeal. It isn’t the kind of thing you write if you want to make people feel better.

Andrew Marantz belongs, unwittingly, to the latter category. The New Yorker staff writer’s new book, Antisocial: Online Extremists, Techno-Utopians, and the Hijacking of the American Conversation seeks to explain “how the unthinkable could become thinkable” — how an extreme right-wing subculture gestated on the Internet and hatched into a full-blown political movement that allegedly memed Donald Trump into the White House. Marantz’s central theory is that this movement was made possible by social-media content algorithms that reward “activating emotions,” such as fear and anger, over truth. We replaced public-spirited old-media gatekeepers with technologists who sought only virality, and political outrage was the surest way to get it. As Marantz says, “bigoted propaganda is great for engagement.”

Like Orwell, Marantz is interested in how far-right extremists achieved so much influence, so quickly. Unlike Orwell, Marantz offers a purely structural explanation, laying blame for those he calls “the Deplorables” with the rules of the online attention economy. The first half of the book is a tour of those rules, demonstrating how a libertarian-utopian culture among Silicon Valley titans — whom Marantz calls “Big Swinging Brains,” BSBs for short — led them to assume that goodness would always prevail if they kept their platforms free from censorship. They spurned their responsibility to keep their platforms safe from “antisocial” messages and subscribed to a morally nihilistic philosophy articulated by one Big Swinging Brain: “The ultimate barometer of quality is: If it gets shared, it’s quality.”

The latter half of the book is devoted to the characters that bloomed in this ecosystem. Marantz is a first-rate journalist whose profiles of alt-right and alt-lite figures — the self-promoting conspiracy theorist Mike Cernovich, the shameful tragedy of Daily Shoah podcaster Mike Enoch, the gothic horror tale of a bright, aimless young girl named Samantha who falls into the clutches of Identity Evropa — are so masterful that time stops when you read them. The trouble begins when he leaves the realm of description and starts theorizing about what needs to change.

For starters, he says, we need a new “moral vocabulary.” Marantz invokes the philosopher Richard Rorty, who believed that the way a society talks to itself through media determines that society’s beliefs. He advised that a transition from one moral vocabulary to another happens roughly the way a paradigm shift happens in science: “Premodern people believed that the sun revolved around the Earth; now everyone, except for a few Internet conspiracy theorists, believes the opposite. . . . A few scientists learned to speak differently about the world, and then, eventually, everyone else learned to speak that way, too.” Marantz often compares the failure to adopt progressive social ideas to the failure to accept scientific truth — his TED Talk advises “contrarian white teens of the world” that being a male-privilege skeptic is like being a round-earth skeptic: They both mean you’re just a “jerk.” For Marantz, jettisoning skepticism of privilege doctrine is as straightforward as jettisoning flat-earthism. We just have to decide to use a different vocabulary. “To change how we talk,” he writes, ‘is to change who we are.”

This makes no sense. It leaves out exactly what Orwell tackled head-on: How do people decide what to believe? Why do they “change how they talk” in the first place? That’s an easy question to answer in the realm of science, because science developed an externally validated set of rules for talking about the physical world. But in the moral world, we validate the rules — which is why concepts such as“privilege” attract such controversy. Why does one moral vocabulary catch fire and another fail? Why, of all the messages that could trigger emotional arousal, did a far-right message find such success, in both the U.S. and Europe? Leftists haven’t exactly been slouches in the historical contest to generate anger and outrage. So why are millions of young men finding themselves down right-wing wormholes on YouTube rather than Marxist ones, or Christian ones, or indeed, flat-earth ones?

Asking that question could lead to uncomfortable answers. Maybe, as Orwell thought, our society is failing to satisfy our core psychological requirements, leaving an identity-shaped hole that the far Right is only too happy to plug. Maybe secularization has been a disaster. Maybe the rules of human sociality place limits on our capacity to absorb mass immigration. Maybe our decision to transform our towns into exurban concrete strips dissolved the possibility of community, and Internet subcultures filled the gap. Maybe family breakdown unmoors young men from healthy masculinity, and maybe prestige-media proselytizing for a “new masculinity” defined by queer, trans, and nonbinary influencers is failing to connect. Maybe decades of neoliberal policymaking inspired working people to hunt around for a political murder weapon. Maybe far-right extremism is a virus well suited to opportunistically exploiting the social failings of our modern age.

And maybe the online far-right subculture wasn’t created ex nihilo by social-media algorithms but emerged out of a dialectic with a new woke subculture on the left — a “response to a response to a response, each one responding angrily to the existence of the other,” as the journalist Angela Nagle argues in her 2017 book on the same subject, Kill All Normies.

Marantz doesn’t think that the formation of an overtly punitive intersectional leftism organized around racial and gender identity, hegemonic in high-prestige cultural spaces but widely despised outside them, has any relevance to identity politics on the right. Which is odd, first because the far Right is called “reactionary” for a reason, and second because the intersectional Left seems to have supplied just the kind of new “moral vocabulary” that Marantz calls for, and it doesn’t seem to have gone over very well.

One senses Marantz’s fear of any angle that could be seen to soft-pedal or both-sides the problem of the far Right — the author repeatedly frets about being morally compromised by merely covering them. The sole time the online Left appears in Marantz’s book is, I kid you not, when he contrasts the Left’s “sincere aspirations to virtue” with the Right’s cynicism — as if being progressive made you immune to social-media outrage incentives.

The next thing Marantz says we need is a new appetite for regulating Internet speech. He treats the need for online gatekeeping as embarrassingly obvious yet is largely silent on how it would work, resorting to metaphor (social media is a party, and sometimes you need to bounce misbehaving guests — something hardly anyone would disagree with) rather than the language of policy and law, which demand concrete line-drawing. He claims this is intentional; it’s also convenient. Marantz takes us inside a Reddit war room where employees make ad hoc, but mostly reasonable, decisions about which communities to ban, suggesting that censorship is easy enough if we deploy the common sense of the average Silicon Valley tech mogul. The problem will go away if we change the rules of the digital conversation, which is an especially attractive solution when the people who control those rules are highly educated fellow elites.

This is why Marantz’s account is ultimately a feel-good story, even though he goes to great pains to reject arc-of-history optimism. For him, dealing with extremism doesn’t mean changing anything about how our society is ordered. Not once does he attribute extremism to social or economic causes that exist outside of the Internet. He acknowledges that extremism is more attractive to people who are “alienated” or “lonely” or who lack “a strong sense of self,” but he doesn’t ask why more Americans than ever seem to feel this way. Instead, he chooses the solutions — a different moral vocabulary and the will to enforce it online — that involve no sacrifice for the class of which he is a member. Indeed, it’s a new privilege — who do we suppose will be teaching us this new vocabulary? Who will enforce its rules? Probably the kinds of people who become New Yorker staff writers.

That’s what we’re all really scared of, conservatives and progressives alike: the other team empowering itself to decide what gets said and what gets buried. Before 2016, social media’s freewheeling speech rules and outrage-stoking algorithms were fine with the Left, because they appeared to amplify the Left’s own voice. People thought the political outcomes of social media would be things like the Occupy movement, or the Arab Spring, or WikiLeaks, or Anonymous — the progressive grassroots speaking truth to power, forever into the future. Anger and outrage directed at the proper target was only social justice. But when the outrage came for the Left, the ruling caste didn’t leave anything to chance. It called upon the class loyalty of its Silicon Valley branch to drop the hammer. The events of 2016 could never be allowed to happen again.

As the power of the new online gatekeepers increases, so will political anger at their every decision — from both sides. Witness the progressive meltdown over Mark Zuckerberg’s decision to take a meeting with a collection of conservative leaders, or Facebook’s attempt to give a fact-checking role to the Weekly Standard. Marantz is nostalgic for the heyday of old-media gatekeepers, but they had power at a time of record-low economic inequality and robust cultural consensus about the boundaries of polite discourse. We are no longer in that time: The cultural consensus was smashed sociologically, by inequality and elite fecklessness, as surely as it was smashed technologically.

For all that Marantz gets wrong, everyone should agree that far-right extremists should have as little influence as possible. But if the Left bypasses root causes and seeks to simply change the rules of the online conversation, the result will be more conflict, not less. Their gatekeeping either will be too tepid for progressive activists or it will enrage the Right, which will hit back — by revoking their protection against liability under of the Communications Decency Act, or with antitrust enforcement against Big Tech, or with a culture war that puts the would-be gatekeepers squarely in the crosshairs.

It’s already happening. The owners of social-media platforms are introducing a new regime of content censorship, the Right is newly interested in war with Silicon Valley, and the world that Marantz calls for is in the process of being born. Perhaps an era of political peace will dawn — or perhaps we’ll soon remember the old wisdom that the only thing worse than no gatekeeper is a gatekeeper who makes everyone angry.


The Latest