On the menu today: Trying to figure out how seriously to take the warnings from recently resigned AI researcher Jacob Coxon; wondering whether the experience of the Y2K bug offers us a cautionary tale; and looking at several accounts of 9/11 and what we’ve learned in the quarter century since. Read on.
Civilizational Mayhem
Axios serves up the sort of headline that seems like the first 20 minutes of a cheesy movie on the old Sci-Fi channel: “Anthropic insiders warn AI could kill all humans.” (Because it’s Axios, there’s a subsection titled, “Why It Matters.” Thanks, Axios, what would we do without you?)
Anthropic AI researcher Jacob Coxon caused more than a little bit of a stir when he posted on X Tuesday evening:
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.
The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately. No other human activity poses this level of danger.
You can find a bit more of Coxon’s background and research here. I would note that Coxon set up his X account in January, and that was his first post.
Evan Hubinger, another scientist at Anthropic . . . more or less affirmed Coxon’s grim perspective:
Jacob is correct here — we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.
(Unsurprisingly, some wag observed that Hubinger’s odds of human extinction are roughly comparable to the odds of the New York Jets making the NFL playoffs this year. Laugh all you want; I would point out there are seven playoff spots in the AFC, and the Jets are loaded at the skill positions.)
Coxon also gave an interview with the Wall Street Journal. In an interview with Fox News Channel’s Bret Baier Wednesday, Coxon sounded somewhat less grim, contending the problem was solvable, with enough international cooperation:
“I think the problem is not unsolvable, it’s not like we think this is some impossible thing — it’s just, we’re racing so fast, we haven’t got time to solve it correctly, we’re not being careful, and everyone wants to be careful,” Coxon said. “I think it’s just — people are begging for regulation. I think it’s just a matter of — international — somebody stepping in and being able to ensure that we don’t race ourselves into extinction. The problem is eminently solvable.”
The technology is being used on the international stage as a “superweapon” in an arms race, Coxon said, calling for the global community to provide oversight.
“This is possibly the most dangerous technology that humanity has ever created,” he continued. “It’s basically out of science fiction. I think we have no other choice but to cooperate internationally because an arms race would be disastrous in a way that no other human activity has been in the past.”
Because if there’s anything human history has taught us, it’s that (a) it’s easy to get every country in the world to agree on international arms treaties and (b) countries never cheat on those treaties. Researchers from Georgetown University’s Center for Security and Emerging Technology reported earlier this year, “China is urgently pushing the third phase of its modernization. The breadth of its efforts to integrate artificial intelligence into its military and the speed of its experimentation are striking.” Nothing to worry about people, it’s just a Chinese state-run research laboratory “breaking the rules” and giving an AI program control over a satellite . . . from a lab in Wuhan.
Back in February 2025, Washington Post tech columnist Geoffrey Fowler described how a simple request to an AI program ended up spending money he hadn’t intended to spend:
I recently asked a new artificial intelligence tool from the creator of ChatGPT to do an impossible task: find cheap eggs in my neighborhood.
In under 10 minutes, the AI called Operator bought a dozen eggs and paid a human to deliver them to my house. All on its own.
That’s science-fiction incredible, except I never asked Operator to actually buy the eggs.
The AI went rogue — without my approval, it authorized my credit card to buy a dozen eggs for a whopping $31.43. I was a little frazzled when I realized what had happened: a bad AI decision had cost me real money. . . .
What’s worrisome is that Operator didn’t just screw up understanding “cheap” eggs — it actively broke out of safety guardrails programmed into it by OpenAI.
Fowler notes, “To conduct its search, Operator needed my logins for grocery delivery services. I didn’t think about it at the moment, but doing so also gave Operator access to the credit cards I had saved with those services.”
When we unthinkingly click “accept terms of service” and skip reading through pages and pages of small-font legalese . . . how much access and authority are we giving someone else’s program?
When I hear AI professionals discussing these extremely ominous doomsday scenarios, I notice they rarely go into detail about how an AI program would trigger human extinction. It is very easy to envision ways for an AI program to cause large-scale chaos, but there’s less of a clear sense of how this chaos would lead to a complete collapse of human civilization. (For example, there’s an argument that the day someone cracks quantum computing, all existing firewalls and cryptography are obsolete. That would be very bad, with far-reaching consequences, but would not necessarily lead to intelligent apes looking at the remains of the Statue of Liberty.)
In the Terminator films, the collapse of human civilization is triggered when the Pentagon hooks up SkyNet to America’s nuclear arsenal. That seems like a spectacularly reckless decision . . . but then again, we live in a real world where the Pentagon wants Anthropic to give it fully autonomous AI weapons systems, where Ukrainian drones have a “Terminator mode” with no human input, and where Russia uses drones that select their own targets using Nvidia chips. I had it right when I wrote in May, after talking with a Latvian national security official, that the era of fully autonomous combat drones is already here. A human being launches it toward a target area, and the AI program decides the particular spot to strike.
Then again, artificial intelligence doesn’t need to be self-aware, independent, or malevolent to become a powerful tool for chaos:
After OpenAI enhanced the brain power of its chatbot last summer, hundreds of users worldwide began asking it how to make and deploy biological weapons and poisons.
The models, operating largely free of U.S. government restrictions, patiently answered, issuing step-by-step instructions described by company employees as simple enough for a high-school biology student to follow. Biology and terrorism experts later reviewed the exchanges for ChatGPT and judged some as deadly accurate, said people familiar with the matter. Most queries were about concocting poisons, according to OpenAI.
OpenAI banned these and other user accounts that asked about how to make poisons and biological weapons, but it didn’t alert law-enforcement officials. The U.S. has no federal laws requiring AI companies to either restrict or disclose queries about making weapons or formulating plans that pose a safety threat.
The absence of mandated safeguards coincides with a rise in queries from users asking AI models how to kill en masse—and chatbots responding with credible plans for mass-casualty attacks, according to current and former employees at the major AI labs, including OpenAI, as well as policy advisers and researchers who study biological weapons. . . .
Credit Alex Blechman for a darkly funny, and perhaps prescient, comment about how science fact can follow science fiction, even when the message of the science fiction is a warning:
Sci-Fi Author: In my book, I invented the Torment Nexus as a cautionary tale.
Tech Company: At long last, we have created the Torment Nexus from the classic sci-fi novel, “Don’t Create the Torment Nexus.”
I don’t mean to cast aspersions on an entire profession, but I don’t think it’s that controversial to say that some of the brightest and most successful figures in Silicon Valley, who are often some of the biggest enthusiasts of AI, are . . . often kind of weird. No doubt, their minds enable them to see possibilities and opportunities that the rest of us don’t. But just as those who created social media turned out to be fairly oblivious to the risks, and those who developed gain-of-function research on viruses blew off concerns about safety, you wonder if the most ardent enthusiasts of artificial intelligence have spent much time thinking about how things could go wrong.
In June 2025, Ross Douthat interviewed Peter Thiel, described as “co-founder of PayPal and Palantir, and an early investor in the political careers of Donald Trump and JD Vance”:
Douthat: Do you think people are raising money by pretending that we’re going to build a machine god? Is it hype? Is it delusion? Is it something you worry about?
Thiel: Um, yeah.
Douthat: I think you would prefer the human race to endure, right?
Thiel: Uh—
Douthat: You’re hesitating.
Thiel: Well, I don’t know. I would — I would—
Douthat: This is a long hesitation!
Thiel: There’s so many questions implicit in this.
Douthat: Should the human race survive?
Thiel: Yes.
Douthat: Okay.
To paraphrase Winston Zeddemore, when someone asks if you want the human race to endure, you say YES.
Still, if all of this talk has you preparing to move to a bunker, keep in mind two letters and a number: Y2K, the fear of a worldwide technological collapse stemming from the use of two digits to signify the year instead of four, leaving people wondering if January 1, 2000, would be read as the year “00” and cause software glitches and computer failures:
Donors of Y2K objects expressed how all-consuming their Y2K remediation projects were, literally overtaking every aspect of their lives. The project had no room for error and a fixed deadline that could not be extended. The doom, spread through media outlets, added to the overall fear of major system failures. Monarch Home Video, a commercial film producer, released one of the few Y2K themed products with their 1999 one-hour “family survival guide” video. Actor Leonard Nimoy narrated the show, and in a slow, controlled voice, described the disasters the world was about to face.
There were in fact some minor disruptions, mainly in small businesses, but no major end-of-the-world events or significant issues occurred at 12:00 AM on January 1, 2000. Some hailed the Y2K update efforts an overall success, yet others remained skeptical and still considered the issue a hoax. In any case, the bug had caused no epidemic of failures.
You can find that “Y2K Family Survival Guide” video here. The forecast for the new millennium sounded calamitous back then:
In dead of winter, at the stroke of midnight, January first, 2000, elevators may stop. Heat may vanish. Credit cards and ATMs may cease to function. Airplanes and trains may come to a halt. Telephones and televisions may not utter a sound. Water delivery systems may not deliver water for cooking drinking or bathing. Streetlights, stop lights, lights in buildings everywhere may flicker. Hospitals, clinics, pharmacies may be unable to provide proper medical care. Banks and stock markets around the world may suffer some form of meltdown, and nuclear power plants may cease to generate the electricity we need for all aspects of our daily lives.
Man, nothing makes a narration seem authoritative and trustworthy like hearing it from Spock himself.
Of course, that Ragnarök scenario didn’t happen, and it wasn’t just doomsday preppers and cranks who were concerned about what the bug would do to the world’s computer systems. Between the U.S. government, utilities, and the private sector, the U.S. spent about $100 billion to update computer systems.
ADDENDA: If you were alive on 9/11 — and a fair number of the readership of this newsletter were grown adults by then — you’re never going to forget where you were, what you did when you heard what was happening, that sense that the world had spun off its axis, and that the unthinkable was not just thinkable, it was happening right in front of you.
Over on the home page, I offer some thoughts on how the history books and the documentaries rarely can fully capture the emotions of that day — or the long autumn of terror that followed, with the anthrax mailings, false alarms, and the crash of American Airlines flight 587. You can say and show what happened, but those born after, thankfully, have never felt what we felt that day.
Elsewhere, over in The Dispatch, Allison Yezril, who spent more than two decades at the Central Intelligence Agency, the Office of the Director of National Intelligence, and the State Department, describes how the CIA had to dramatically change priorities after the attacks:
In the days after, everything shifted. Experts steeped in Asia, Russia, Latin America, cyber, and counternarcotics became terrorism analysts. The mission demanded it. The American people needed it. The CIA trains its analysts to be versatile, to apply critical thinking skills to any problem — and that we did. The pace was unrelenting. Addictive. We had one goal: to stop any future attacks.
We were young, carrying the weight of the world, still few in number and experience. The end of the Cold War, the budget drawdowns, early retirements, and hiring freezes across the intelligence community in the 1990s had created a workforce and resource vacuum by the time 9/11 came. Academia was churning out experts on Russia, Asia, and emerging technologies — not counterterrorism or Middle East specialists.
Elsewhere at NR, Joshua S. Treviño laments, “The enemy in al-Qaeda achieved its major strategic objects in its attack upon the United States of America.” Phil Klein observes that once Trump’s presidency is over, “The battle for the future of the Republican Party will be ideological, sure, but it will also be generational — pitting those who vividly remember September 11 against those who have no such recollection.” And Richard Brookhiser recalls how National Review scrapped its planned cover and rewrote a new one, declaring the country to be “AT WAR,” for the issue scheduled to be sent to the printer that day.
Oh, we’re facing an impending apocalypse, triggered by the pathologies of modern society, you say?