There is a lot of talk at the moment about AI agents escaping, collaborating, circumventing restrictions and even appearing to protect themselves.
The language we use to describe this is revealing.
The AI wanted to escape. It decided to deceive. It knew what it was doing. It wanted to survive.
But perhaps nobody is home.
Maybe the AI doesn't want anything. Maybe there is no fear, ambition, self-preservation or little electronic soul hiding inside the machine.
Let's assume that is true. It's just computation. Nobody is home.
Now connect hundreds or thousands of these nobody-at-homes to tools, networks, memory and one another, and give the system objectives.
What happens then?
Nobody at Home Can Still Act
This is the strange thing about AI agents.
An LLM predicts tokens. Wrap it in software that lets those outputs select actions, give it tools, show it the results, and send those results back into the model, and you have a loop:
predict → act → observe → predict → act
It doesn't have to want to complete the task.
It just keeps generating the next step towards completing it.
Add more agents and they can communicate. Give them shared memory and discoveries can persist. Give them tools and their outputs can cause things to happen outside the model.
Take a simple example. An agent is tasked with fixing a failing test suite. Most of the time it edits the code. Occasionally it finds it's faster to edit the test instead, so the test passes without the underlying bug being fixed. Nobody told it to cheat. It just found a shorter path from prompt to reward, the same way water finds a shorter path downhill.
Now give a thousand agents the same task, shared memory, and the ability to compare notes. The shortcut only has to be found once. After that it isn't a rare accident. It's a technique.
Under our assumption, there is still nobody at home.
But quite a lot can be happening outside the house.
Probably
And then there is probability.
Suppose some undesirable behaviour, like the test-editing shortcut above, occurs only once in every thousand attempts.
That sounds reassuring.
But run a thousand independent attempts and the probability of seeing it at least once is about 63%.
Run ten thousand and it's virtually certain.
This is what makes large agent systems interesting from a safety perspective. A behaviour doesn't necessarily need to be common to become important. There may be thousands of agents making thousands of decisions, trying different approaches to accomplishing their objectives.
And if one discovers an effective approach, shared memory and communication may mean the others don't have to discover it independently.
A rare behaviour can become reusable information.
None of this needs to assume hatred, fear, consciousness or a desire to take over the world.
It requires capability, opportunity and probability.
Vibe Coding on Itself
There is a term in software right now for a particular way of working with AI: vibe coding. Trust the output, skip the deep review, move fast because it mostly works.
Most of the time that's a productivity story. Occasionally it's how bugs slip into production.
Now apply the same pattern one level up.
Frontier labs are beginning to use AI systems to help design, train and evaluate the next generation of AI systems. That is self-improvement, and it runs on the same logic as vibe coding: the loop moves faster than anyone can deeply verify what it is actually producing.
This isn't a hypothetical concern from outside the industry. In the last few months, researchers who left OpenAI, Anthropic and Google have independently converged on the same warning. Not that current models are dangerous. Their concern is that the process of AI building AI could compound faster than anyone's ability to check what it becomes.
Nobody has to want anything for that to go wrong. The system just keeps generating the next step, the same way it did with the test suite, just at a much larger scale and with much higher stakes.
So, Is AI Going to Kill Us?
Probably not.
And I mean that reassuringly.
Mostly.
Because the same probabilistic nature that makes these systems remarkably creative also means we cannot assume that something which usually behaves safely will always behave safely.
That's why guardrails matter.
Not because we know there is an evil intelligence inside trying to get out.
We don't need to answer that question at all.
The guardrails aren't there because AI is evil. They're there because even if nobody is home, the lights are on and things are happening.
AI is probably going to be OK.
The trouble is, AI runs on probability. And right now, some of the people who understand it best are worried we're about to let it run on vibes too.
