So. It’s been a bit of a week, hey?
I don’t know about you, but I’ve been elbows deep in escalating doomerism, after the latest warning from an AI company defector set off a new flurry of “they’ll kill us all!” handwringing.
Outgoing Anthropic researcher Jacob Coxon told anyone who would listen that OpenAI and Anthropic were “racing straight to self-improving superintelligence and gambling with our lives.”
But you might have noticed very few of the “OMG they’re coming for us” narratives properly explained exactly how they would do so.
And that’s important, because if we don’t understand the actual risk scenarios, it’s pretty hard to take them seriously — let alone agree on some actual, proactive prevention.
All the ways people imagine this would come about are pretty sci-fi. That’s kind of the problem with tech-apocalypse. We’ve consumed so much pop culture about it at this point it sounds silly to suggest it could really happen. But as you’ve probably read this week, some serious people rate the robotic end of days as a non-zero possibility.
So without further ado, I present The Tokenised Human guide to robo-apocalypse lore:
That’s your basic rundown of the leading end of days scenarios. But the harder question is what these grim possibilities reveal about power and control.
That is one of the many topics I spoke about with UQ’s associate professor Michael Noetel in the first episode of the Tokenised Human podcast (it launches tomorrow, September 18!).
“We did a study recently trying to get AI experts to prioritise the risks from AI,” he tells me. “And some of them were things that you and I probably heard about a lot. Like, maybe AI could do big cyber attacks or create biological weapons. But one of the top things that the experts flagged was power concentration, which is where one person in the Broligarchy could effectively be a dictator. Actually every AI company was started because they didn’t want the other guy to be the dictator.”
Delightful.
Anyway, I am really pumped for you fine folks to hear the pod.
The first season is six episodes, in your feed on Friday mornings. You can listen inside the Substack app — if you’re a subscriber you’ll get an extra Friday email showing you where to go — or subscribe now on Spotify, Apple, or wherever else you prefer to listen.
If this letter gave you something to think about, consider forwarding it to one person who’d enjoy it too! Word of mouth is how a newsletter this new finds its people.
And I’d love to hear from you — reply to this email, comment in the app, or find me on LinkedIn or Bluesky.
OMG I love this so much! Can't wait for the podcast 🤖