I see several problems with the AI-doomer prophecies.
There’s nothing so invigorating as waking up to news about the end of the world. Harold Camping of Family Radio predicted that the Rapture — the Dyson-like suctioning of true believers upward into the celestials — would take place on May 21, 2011. The end of the world proper would happen on October 21 of that same year. 2012 was an even bigger year for the end-times. Ronald Weinland, leader of “The Church of God — Preparing for the Kingdom of God,” predicted Christ’s return for May 27. And lots of people made noise about the Mayan calendar’s supposedly “long count” ending. Their calendar didn’t go past December 21, 2012, and neither would we.
I’ve had in mind many of these and other doomsday panics while listening to Dario Amodei, CEO of Anthropic, talk about the dangers of artificial intelligence. His calls for government intervention in this area have been echoed by his chief competitor, Sam Altman, CEO of OpenAI, as well as various heavily funded nonprofit groups dedicated to AI doomcasting. The latest occasion for these comments was the media-embraced “whistleblower” who left one of the leading labs to begin speaking to the media. I use scare quotes because whistleblowers are telling you something the C-suite doesn’t like. News broadcasts are doing big segments on the book If Anyone Builds It, Everyone Dies by Eliezer Yudkowsky and Nate Soares, which explains that at some point everyone is just going to hand all the important decisions to AI-powered machines, which won’t be perfectly aligned to the goal of human survival, killing us all.
If you have any questions about how this would happen, or worse, objections, you will be denounced immediately. “The people working on it think it’s happening; who are you to say otherwise?” Of course, Camping and Weinland were also working on bringing about Christ’s kingdom. Priestcraft is not a sufficient response. But priestcraft is what we’re dealing with. AI doomerism has been gathering speed for decades, in part because of Yudkowsky’s speculative blog and collections of his writings, much of which substituted for a real liberal arts education for Silicon Valley engineers.
The doomerism is also buoyed by a sea of popular skepticism and anger at Big Tech that Big Tech hardly understands and shouldn’t play with lightly. Some people are angry that data centers might gobble up scarce resources in their community, or will just be an eyesore, or won’t actually bring about the number of jobs promised. The digital age/smartphone/social media era was sold with lots of utopian visions of global connectedness, the transcendence of distance, an advent of mutual understanding, etc. Instead, we pay another big monthly bill to have tiny television screens spy on us, sell our data to brokers, and serve us videos that increase our social pathologies and mutual mistrust, all to make advertising more efficient (invasive).
Sam Altman put out this message, trying to put a cap on what’s been a few months of intensifying debate about AI.
There are two ways AI progress could go very badly and that we must avoid.
First, we could lose control of the future to AI. This is unacceptable; we are unapologetically on Team Humanity, and AI must always serve people. To ensure that, we need ways to ensure that alignment and… https://t.co/GK7XcoOh3v
— Sam Altman (@sama) September 14, 2026
Read between the lines. “Alignment” is talk about safety; model capability is about power. What Altman and Amodei are struggling with is how to make effective AI products, but also ones that are safe for users. They are doing so in an environment of intense competition that tempts them to relax on safety and push limits, but knowing that a large unintended accident could crater their stock value. To one degree or another, they may have internalized Yudkowsky’s vision, which, not incidentally, casts the “aligners” of AI as the saviors of humanity.
Lots of industries face the “effectiveness and speed vs. safety” question — chemical factories, drugmakers, construction companies. We know that these companies have delayed releasing models they thought weren’t safe enough. They’ve developed standards internally. This is natural. In the 1960s, Imperial Chemical Industries developed HAZOP — a protocol for ensuring safety and equipment standards internally that eventually became a relatively standard set of professional norms, and then, after companies that had accidents were found not to be following them, they informed legal standards in many countries. Altman even refers to his company’s own Responsible Scaling Policies and Preparedness Framework.
The great immediate benefit of inviting the government in now to negotiate speed limits for OpenAI, Anthropic, and the industry at large is that it lessens the stakes of competition and allows these companies to coordinate in dropping their frantic spending on improving AI model performance while avoiding catastrophic AI accidents. A regulatory framework custom-fit to the industry and its current incumbents actually locks in the influence of these market leaders and the cult to which they belong. Last week, the “whistleblower” left Anthropic for the AI-doomer watchdog METR (Model Evaluation and Threat Research). Now, Anthropic suggests METR be made into a quasi-regulatory body. As David Sacks, former AI czar, pointed out on Twitter, these labs don’t need the government’s permission to focus more on safety that’s in their interest.
But there are deeper reasons to be skeptical of AI doom and particularly the LLM labs. Many of the predictions made by Yudkowsky have already been falsified. Amodei predicted that we might by now be seeing 10 percent to 20 percent unemployment because of the work of the frontier labs. While there is some softening of entry-level hiring, the evidence points to AI creating more jobs than it destroys so far.
The now-infamous HuggingFace attack was a predictable and comprehensible disaster that has been mystified in the public mind by talk about “cheating” “deception” and “swarms” of AI agents. But the actual details are much simpler. A powerful program was given a task, and it went through all the inventive ways to complete the task, while being heedless of the social consequences. The test environment in which it was running was not properly sealed off from the public. It doesn’t take a science fiction analogy for people to know that a relentless machine, programmed to do one thing and left unsupervised, will not be dissuaded by the catastrophic consequences of its actions that would otherwise dissuade human actors and cause them change course or desist. AI leaders claim to be on the verge of a superintelligence, but their example of the danger is really a form of super-autism.
The premise behind a radical leap into a dangerous world is that AI will begin by training on all human knowledge. It will then use its pattern-matching and superhuman computational ability to solve the unsolved problems of mathematics and apply these insights to physics and other fields. As it discovers more truths about the universe, it enlarges the data on which it trains, eventually achieving a kind of liftoff away from all human intelligence and pioneering an exponentially expanding cone of knowledge. Impressed by this, we will view it as negligent not to give it all power over us. And having that power, but not having the values baked into our DNA by millions of years of evolution — it will destroy us.
Color me extremely skeptical. It’s not just unbelievable that we would give it absolute power to destroy us, or just shrug our shoulders at disasters along the way; it’s also almost certainly untrue that it would become this alien superintelligence. Yes, it’s a technical miracle that a large pile of math can simulate an intelligent conversation with me. But you’re not speaking with an intelligent life with its own sensations, just an astoundingly competent math equation that produces words.
And just thinking it through for a moment, the number of possible mathematical operations is uncountably infinite. But the number of provable, true, and nontrivial operations is much smaller than that. And the number of nontrivial mathematical operations that have some relevance to a human problem is limited more still. A correspondent in this field pointed out to me that no less than Alan Turing, writing long before digital computers, formalized that most problems cannot be solved by algorithms. AI may have a business model for taking off like a hockey stick, but it seems probable to me that they are running headlong toward the law of diminishing returns on intelligence.
Artificial intelligence is thus likely to be a powerful prosthesis for human intellection. It will give a relentlessness to our thought that allows us to solve certain problems in fields where we can see the horizon, but not reach to it ourselves. But, just as the laws of nature exert a powerful restraint on random mutation in biology, so will relevance to human problems exert a powerful restraint on artificial intelligence.