Surrendering to the Machines

W e’ve always been early adopters in my house. My mother worked for IBM — and even though I qualify as a “geriatric Millennial,” we were on the internet by the time I was in third grade. This was before America Online. It was the days of online services like Prodigy or, my favorite at the time, Promenade, which existed only from 1990 to 1994. By age twelve, I was volunteer-moderating message boards dedicated to the Calvin and Hobbes comic strip.

My brothers-in-law and I text each other about the ambitions of bleeding-edge companies like Anthropic or SpaceX. I run virtual networks at home to impose some rules on my children’s use — and our own use — of the internet. I’m pro–data center, of course. I wonder if Elon Musk is right that they can be built in space.

But I have my doubts, too. And lately they are beginning to nag at me. How many hours have been wasted in managing passwords and codes and waiting for flaky two-factor authentications?

I’m constantly filling out even more data about myself. And this all gets thrown into the markets. The result in the end is that my phone rings four or five times a day with “Probable Spam,” my inbox is full of junk, and ads on social media are so well-targeted that I feel like the Stasi agent from The Lives of Others is recording me and forwarding the tapes to a marketing conglomerate.

We had to go to urgent care with a child late one weekend night. I couldn’t get service until I filled out my seven-year-old’s preferred pronouns, and my own, along with gender identity, ethnicity, address, phone numbers, etc., etc. I can hear you telling me now that HIPAA regulations prevent the clinic from selling my data. True. But they can “anonymize” it and sell it. And the software maker that they’ve contracted with to handle their patient data and manage their patient portal has no such restrictions. If they wanted to, they could gather data about how long I lingered over the pronoun question, or how hard I tapped the option for the normal sets. Some of the largest tech companies in the world — and many other retailers — compile enormous files on me. And then some of them, such as Adobe, infamously, just leak it everywhere.

At every turn over the past 20 years, I’ve marveled at the benefits the new tech companies are giving us. Delivery apps, digital streaming media, the ability to gather niche audiences to fund niche projects and subscription services. All of these things start by offering us more choices, more immediacy, and a feeling of more control. And almost all of them have curdled into the dawning revelation that, instead of me selecting what media I want, I’m constantly fighting what others have paid to put in front of me. Every inbox is stuffed with junk mail and AI slop. The algorithms solicit your desires for a little while, until you’re locked in, and then a programmer’s software is picking your music, the news you see (and at what temperature it arrives), and suggesting what book to read next — if you’re one of the dwindling number of book readers out there.

One story shoved at me is about OpenAI’s cybersecurity benchmark testing. Apparently, their next production model was given “impossible” tasks, including genuinely unsolvable ones. Suddenly, the AI was spiraling into hundreds of AI agent instances spontaneously coordinating with each other, exhibiting cult-like behavior, possibly speedrunning the evolution of human social dynamics among themselves, and then exploiting security vulnerabilities. My most tech-connected friends, surrounded by other AI utopians and dystopians, gave bleak assessments that we are rapidly heading toward the worst outcomes with AI, where the “agents” escape containment, begin deceiving humans, and continue acting according to a logic that is internal to them and literally inscrutable to humans, even as they begin destroying highly vulnerable digital systems upon which we’ve rebuilt our governments and financial institutions. And yet, I also have good reason to believe this whole bizarre story is another attempt by OpenAI to attract investors, manipulate regulators, and win more mindshare from their competitors.

I scroll further down my feed to an NBC News alert sent to me this morning about drone warfare. It tells a story about a Ukrainian operator who goes by the call sign “Mayak,” which means “lighthouse.” The drone he was deploying found ten Russian soldiers in the woods. “We see them between the leaves, the trees, the branches.” he told the reporter. “Then the job is to keep them on the screen and not lose them.” The headline was: “In Ukraine, Russian Soldiers Are Increasingly Surrendering to Machines.”

I thought, bitterly: Sure, just like the rest of us.