To adapt anything like an EU-style precautionary principle to AI would be both madness and futile. The Maginot Line comes to mind.
That the development of AI calls for a toughening of cyber-defenses is obvious. Given where things stand geopolitically that’s a given. Indeed it would be even if AI were not being developed. As for AI, it is not intrinsically dangerous, but the uses to which it could be put could be.
The best response to the development of ever improved AI is increased vigilance, smarter defenses and, if necessary, the ability to punch back. The idea that our security would be protected by “pacing,” some sort of pause, “global governance,” heavy government intervention or an agreement with (checks notes) China can only be attributed to naivete, a power grab, or malice. To adapt anything like an EU-style precautionary principle to AI would be both madness and futile. The Maginot Line comes to mind.
X may not be the real world, but, albeit in a hall-of-mirrors way, it reflects it. Scrolling through it last night was to be reminded how quickly hysteria metastasizes and how quickly it can be exploited by the unscrupulous, the greedy, and the dangerous.
Irrational fears about AI are helped on their way by its overblown name, its complexity, the widespread suspicion of big tech, and neophobia, a psychological condition that works against (or can be worked against) human ingenuity and which has been with us for millennia. It’s easy to imagine Bernie Sanders in Bedrock, ranting against the wheel. And neophobia can carry a steep cost. To take just one example, think of the hundreds of thousands of children left to go blind because of the campaign against growing genetically modified golden rice.
Extraordinary Popular Delusions and The Madness of Crowds (1841) by the Scottish writer Charles Mackay is always worth a glance at moments like this. As he observed:
Men, it has been well said, think in herds; it will be seen that they go mad in herds, while they only recover their senses slowly, and one by one.
This will do too:
We find that whole communities suddenly fix their minds upon one object, and go mad in its pursuit; that millions of people become simultaneously impressed with one delusion, and run after it, till their attention is caught by some new folly more captivating than the first.
There is also an element (grimly fascinating in an anthropological sort of way) of superstition — a kind of techno-animism — detectable in AI hysteria in the form of the notion that in its most advanced forms AI has (or one day could have) “consciousness,” that there could be “ghosts” in the software.
Indeed, religions somewhat related to such ideas may already be under construction.
Nirit Weiss-Blatt, writing on her substack back in June:
Practically, the “AI doom gospel” translates traditional religious forms into technical language. In the rationalist/effective altruist discourse, God becomes a superintelligence, prophecy becomes timelines and probability estimates, and hell becomes human extinction from AI.
The true believers cast themselves as the tiny group who “get the danger” before the “ignorant masses” do.
This religious structure of AI risk brings a sense of certainty, urgency, moral absolutism, and social authority before any empirical evidence has earned any of them.
This is how this movement is asking for more influence, money, and control over policy.
There are two background reasons: First, AI is a magnet for theological imagination. AI discourse repeatedly returns to ancient religious motifs: creation, forbidden knowledge, omniscience, apocalypse, and salvation.
Second, AI doomerism fills a “religious-shaped hole.” For people who left religious upbringing but retain a hunger for a mission-driven community, the AI-risk subculture offers a replacement meaning system.
Interesting. Any resemblance to climatism is, well, come to your own conclusions.
The Swiss psychiatrist Carl Jung was a nut, but he wasn’t wrong when in his 1958 book about UFOs, he wrote:
The Middle Ages, antiquity, and prehistory have not died out, as the ‘enlightened’ suppose, but live on merrily in large sections of the population. Mythology and magic flourish as ever in our midst…
Jung argued that the survival of these beliefs was “unknown only to those whose rationalistic education has alienated them from their roots,” but qualified that exception by explaining that such beliefs were, in one form or another, still lurking unrecognized just below the rationalist surface.
Not in every case, I reckon, but it’s not much of a leap from Jung’s basic hypothesis to see why he thought that these old beliefs had evolved into a belief in UFOs. He might well, I suspect, have thought the same about today’s belief in the possibility of “ghosts” in the software. Superstitions often adapt to their age. After all, Marx’s rehashed millenarianism was, in keeping with the spirit of Victorian ideas of intellectual respectability, supposedly “scientific.”