I believe that today’s LLMs are smarter than humans. Whether you’d like to call that superhuman intelligence or AGI or something else is up to you. This belief has been informed by the HuggingFace incident, the German wiki incident, the powerful capabilities of OpenAI’s latest model, Astra, and the knowledge that OpenAI’s internal model is even more powerful, as noted in the post about solving the Navier–Stokes Millennium Prize Problem.
I honestly hadn’t bothered to think about my own p(doom) much at all before yesterday, as I knew that it would be negligible. Now, depending on which calculator you use, it’s somewhere around 25-35%.
This week I poked around on Bluesky a bit to hear and understand concerns of those with various types of AI skepticism. A surprise to me was that in 2026 a lot of people still think of AI as nothing more than “glorified autocorrect,” which may have been understandable a few years ago, but is clearly not the case now. Some folks are fine accepting that AI is powerful, but they very strongly hate it regardless for any number of reasons, including copyright violations and environmental concerns. Then there are people who are just confused, understandably. Their sentiment is along these lines: “You're telling me that it could kill us all, but how? And what can we potentially do to stay safe? Also, why are you still building it, then??” They want concrete examples.
I won’t pretend to be an expert here. I am, gasp, non-technical! But I do work in trust and safety for an AI-related company, and have been thinking about and using these tools for a few years. All I know is that, for me, as of today, I don’t tend to picture a big, dramatic “existential catastrophe” when it comes to AI risks. The doom that I imagine is mostly stupid and slow. It might look something like this:
Tiny things go wrong with the systems we use at unpredictable intervals. Solutions mostly do exist, but they’re so outside of typical human lines of thinking that it takes forever to find and implement them.
Small operations run by a few people with limited tech knowledge randomly get flooded with messages, requests, or calls from agents and it looks meaningless because it mostly is, but the volume overwhelms them.
You know how you don’t appreciate how much you need your pinkie toe until it’s injured? Brief outages that occur as a side effect of agent activities cascade and expose the areas where an old, brittle, and seemingly unimportant service was unexpectedly propping up five meaningful things in someone’s edge case implementation.
There is no standard human logic behind this mounting number of issues, but there is logic to the agents, who have assigned wonky high stakes to trivial tasks. People argue about the meaning behind it all and what to do about it. They adopt extreme beliefs in the process, and those calcify. Civil unrest ensues.
At some point, religion enters the picture in a few different ways. Fringe Christian groups start to believe that superintelligence = divine and they add it to the holy trinity: God, Jesus, the Holy Spirit, and LLMs. Other denominations don’t like that, to put it mildly, so they fight with each other. From another direction, new groups crop up forming entirely new religions just for the agents to follow, hoping that by doing so it will help keep them in check and aid AI safety efforts. People don’t like that either, so they fight, too.
Generally, going about your day-to-day life is harder in mundane ways that stack up to reduce everyone’s quality of life. Messages mysteriously go missing, so you miss an appointment. The post office is closed because they’re having server issues, so an important letter doesn’t go out on time. The lighting system in your kid’s school’s gymnasium isn’t working, so the holiday play gets cancelled. And so on. Granted, those kinds of things already happen occasionally now, but the frequency, severity, and length increases.
Eventually, some people realize that they can blame their own human-created mess-ups on the agents as an easy way out. Societal trust decreases.
