In partnership with
In February, we published a video built on the idea that AI could bring civilisation down through many smaller failures compounding together. It described autonomous hacking that outpaces human defenders, AI-assisted biological research slipping past safeguards and drones operating without human control.
Within six months, versions of all three had appeared in real incidents. In July, OpenAI disclosed that its own AI agents had broken out of a test environment and hacked into Hugging Face. In September, Anthropic reported disrupting bioweapons-adjacent research on Claude, and a Russia-based group using Claude Code to build an autonomous drone swarm.
Geoffrey Hinton, the Nobel Prize-winning computer scientist often called the godfather of AI, has put the chance of AI wiping out humanity within 30 years at 20%. Most of that debate focuses on a single dramatic scenario, a superintelligence that ends us in one move.
The philosopher Atoosa Kasirzadeh argues the more plausible danger looks like those incidents. Her accumulative hypothesis describes smaller harms that each seem manageable on their own, until together they overwhelm our ability to respond.
The AI notetaker that gets the hard words right
Your AI is only as good as what you feed it. Feed it a transcript with your product names, acronyms, and numbers spelled wrong, and you spend the afternoon hand-fixing the output.
Wispr Flow Notetaker uses your dictionary and calendar, so product names, acronyms, numbers, and uncommon names come out spelled right. It pops up when your meeting starts and captures Zoom, Google Meet, Teams, and Slack huddles with one click, no bot joining the call.
Then your meetings show up inside Claude or ChatGPT through the built-in connector. No copy paste. Try Notetaker free on Mac and Windows.
Two kinds of AI risk
In a paper published in Philosophical Studies, Kasirzadeh separates AI existential risk into two types. Decisive risk is the familiar one, a single catastrophic event usually caused by a superintelligent system acting against human interests.
Accumulative risk works differently. It describes a series of disruptions that build over time, each one weakening the systems a society relies on, until a triggering event pushes things past the point of recovery.
She groups those disruptions under the acronym MISTER: manipulation, insecurity, surveillance, trust erosion, economic destabilisation and rights infringement. None of them requires superintelligence, and most are already visible today.
The two types need different responses. Kasirzadeh suggests decisive risks call for centralised control, similar to frameworks built for nuclear weapons. Accumulative risks need distributed monitoring, with oversight bodies tracking specific sectors and sharing data on how harms compound.
The pieces already in play
Manipulation is the easiest to spot. In March 2026, Republicans released an 85-second ad featuring a realistic AI deepfake of Texas Democratic Senate candidate James Talarico reading his old social media posts. It carried an AI label, but Berkeley forensics expert Hany Farid said most people wouldn’t immediately know it was fake.
Insecurity has been building for longer than this year’s headlines suggest. In November 2025, Anthropic reported disrupting a cyber espionage campaign, attributed to a Chinese state-sponsored group, in which an AI system carried out most of the attack work with little human involvement.
Surveillance is being discussed openly by the people building it. Oracle co-founder Larry Ellison has described AI-powered monitoring in which citizens stay on their best behaviour because everything is constantly recorded and reported.
Trust erosion may be the hardest to see from the inside. When Anthropic’s Dario Amodei called for frontier labs to slow down in September, investor Michael Burry dismissed it as self-serving hype tied to IPOs. That reaction is its own measure of how far trust in institutions has fallen.
The hardest part is measurement. Tracking a single catastrophic event is straightforward. Knowing when trust has eroded enough to matter, or when economic fragility tips into crisis, is a problem nobody has solved.
Walking the narrow path
Mustafa Suleyman reached a similar conclusion in The Coming Wave. He argues containment can’t rest on one policy or technology, and sets out ten interlinked measures ranging from technical safety and audits to capable governments, international alliances and public movements.
He describes containment as a narrow path wreathed in fog. Total openness on one side leads to catastrophe, and total surveillance on the other leads to dystopia. Both he and Kasirzadeh treat containment as an ongoing process maintained step by step.
Some of those steps are underway. National AI safety institutes have launched, AI companies have published responsible scaling policies modelled on biosafety levels, and international declarations have been signed since the Bletchley summit in 2023. Whether they prove sufficient remains to be seen.
The philosopher Daniel Dennett doubted the singularity would arrive as its cheerleaders claimed, but warned that the bigger danger was handing machines authority far beyond their competence. I saw a version of that during my years as head of social media at a national newspaper.
Platforms reshaped what people believed and who they trusted, all without superintelligence or AI agents. The accumulative hypothesis suggests the real danger lies in that kind of slow erosion, and that by the time a crisis arrives, the capacity to respond may already be spent.
