I’ve just dug up my old photocopy of John Searle’s Minds, Brains and Science, the 1984 Reith Lectures, which I copied at university in 1998.
It’s the only book from that time I’ve still got, carried from the UK to Singapore and through every place I’ve lived since. I kept it because something in it never quite left me, particularly its refusal to let me take understanding for granted. Most of what I studied that year has long since faded. This one hasn’t.
Delivered as the BBC Reith Lectures in 1984, it asks a question that suddenly feels remarkably current: if a computer produces the right answer, does that mean it understands the answer?
Searle’s answer was no.
His famous Chinese Room thought experiment imagined someone who knows no Chinese manipulating Chinese symbols according to a set of rules. With sufficiently good rules, their answers could become indistinguishable from those of a native Chinese speaker.
But the person still wouldn’t understand a word of Chinese.
I have a confession that makes this less academic than I’d like. My son is learning Chinese, and homework duty sometimes falls to me when his mother’s out of the house. I don’t speak Chinese, despite twenty-three years in Singapore. So I scan his Chinese homework into ChatGPT, which turns the characters into English I can actually explain to him. I am, quite literally, running symbols through a system that manipulates them correctly without either of us fully understanding them, and then presenting the output as help. Searle would have opinions about my parenting.
Searle’s distinction was essentially syntax versus semantics. Correctly manipulating symbols does not, by itself, establish that those symbols mean anything to the machine manipulating them.
We’ve actually been comfortable with this distinction for years.
An algorithm shows you exactly the trainers you were thinking about buying. Spotify recommends precisely the music you like. You joke that “the algorithm knows me.”
But nobody gets worked up and declares that the advertising algorithm is alive. We understand that “knows” is a convenient metaphor for some very impressive pattern matching and prediction.
Then the machine learned to talk.
Suddenly our language changed.
The AI knew it was being tested. The AI tried to deceive the researchers. The AI wanted to avoid being shut down.
We started turning useful metaphors into claims about minds.
Not everyone accepted Searle’s conclusion, even at the time. The most persistent objection, the “systems reply,” which Searle anticipated and tried to head off in the original paper, argues that of course the man in the room doesn’t understand Chinese, but maybe the system does: the man, the rulebook, and the paper together, as a whole, might understand even though no single part does. Searle’s response was to internalise the entire system: imagine the man memorising the rulebook and the data, so there’s supposedly nothing left outside him to which understanding could be attributed. Whether that actually answers the objection or just relocates it is something people are still arguing about. I’m not raising it to settle it here, but because any argument that treats “no understanding” as obvious is skipping past forty-five years of people arguing about exactly that.
None of this means modern AI isn’t extraordinary. Searle himself distinguished between computers as powerful tools and the much stronger claim that running the appropriate program literally constitutes a mind.
And today we can go considerably further. Connect an LLM to memory, tools, networks and feedback loops and it can do things in the world.
But as a first approximation, there are three different steps here:
CAPABILITY → AGENCY → INTENTIONALITY
Capability means it can do something.
Agency means we have built a system that can pursue objectives through actions.
Intentionality means the system’s states are genuinely about or directed at things in the world, rather than merely functioning as if they are.
We shouldn’t assume that demonstrating one automatically demonstrates the next.
A thousand rooms
But that sequence gets more interesting, not less, once you stop imagining a single model and start imagining many of them working together. Because that takes you somewhere that is, in effect, a modernised systems reply to Searle.
Imagine a thousand agents. Each one runs an LLM and has a narrow role: one searches, another checks evidence, another plans, another criticises, another remembers previous attempts, another observes what happened in the world. They communicate, revise one another’s conclusions, and maintain shared memory.
Perhaps none of the individual LLM calls understands anything in Searle’s sense.
But the interesting question becomes: could understanding be a property of the swarm rather than of any single model in it?
There’s a human analogy here. No individual neuron understands Searle’s paper. Yet the organised system of billions of neurons apparently does. Pointing out that a component lacks understanding doesn’t, by itself, establish that the larger system lacks it too.
This isn’t entirely new ground for Searle, either. His critics already pushed him on something adjacent, the “robot reply”: put the computer inside a robot with cameras and limbs, so it can perceive and act in the world, and see if that’s enough for intentionality. Searle still rejected it. Embodiment alone didn’t move him.
But a swarm goes considerably further than that hypothetical robot. It’s not just perception and action bolted onto a single symbol-manipulator. It’s distributed cognition: specialisation, criticism, shared memory, and persistent recursive interaction between many agents and the world, repeated at scale. You could have a loop like this, repeated millions of times:
LLM → action → world → observation → memory → other agents → criticism → revised model → new action
A system running that loop could, in principle, develop stable internal representations of things outside itself, discover when those representations are wrong, revise them, retain knowledge across time, and use that knowledge to accomplish new objectives.
This idea has a history of its own. In 1986, Marvin Minsky’s The Society of Mind proposed something strikingly similar in structure: intelligence emerging from the interaction of many simpler, individually mindless agents. The interesting properties belonged to the organisation, not necessarily to any one component.
And now we’re beginning to build systems where this is more than a thought experiment.
In 2026, researchers including several at Tsinghua University tested 108 different groups of LLM agents, varying their size, the models involved, and how the agents communicated. They found evidence of a measurable “Artificial Collective Intelligence” factor that helped predict how well the groups would perform on new tasks.
An even more provocative phrase comes from a 2025 paper by Domenico Maisto, Davide Nuzzi and Giovanni Pezzulo: what the flock knows that the birds do not. Their agents weren’t LLMs, and the research certainly didn’t demonstrate consciousness. But the researchers found that interacting agents could form a higher-level system exhibiting joint agency and collective information about its environment that wasn’t available to every individual member.
That’s an important distinction. It doesn’t show that anybody is home. But it does suggest that examining every room individually may not tell you everything about what’s happening in the building as a whole.
At that point it’s worth separating three different claims that are easy to blur together:
- Functional understanding: the system has sufficiently rich models of the world to reason, predict, correct itself, and act appropriately.
- Intentionality: those representations are genuinely about or directed at things in the world, rather than merely functioning as if they are.
- Consciousness: there is actually something it is like to be that system. There is subjective experience. Somebody is home.
A large enough swarm might make the first increasingly hard to deny while leaving the second and third unresolved. And even those last two need not stand or fall together.
Which gives the earlier sequence its full shape:
CAPABILITY → AGENCY → FUNCTIONAL UNDERSTANDING → INTENTIONALITY? → CONSCIOUSNESS?
That leaves room for a stranger possibility than the flat “nobody’s home.” The swarm might eventually qualify as understanding its environment in a functional sense. We might even have reason to ask whether its representations count as intentional. And yet there could still be nobody home phenomenally. Understanding, aboutness, and experience may not be the same achievement at all.
Is anybody home in the building?
Perhaps nobody is home in any individual room. Each agent, taken alone, may still be doing something analogous to Searle’s symbol manipulation, however sophisticated.
But now connect a thousand rooms. Give them different jobs. Let them communicate. Give the system memory. Let it observe the world, act upon it, discover that it was wrong, change its representation of the world, and try again.
Searle considered something related to this in his robot reply, and he still didn’t think embodiment alone established understanding. But we’re now contemplating something stranger: distributed systems of models, tools and memories continuously interacting with each other and with the world, at a scale and with a persistence his thought experiment never had to account for.
Perhaps that’s still just a much bigger Chinese Room. Or perhaps we’ve reached the point where the old systems reply becomes considerably harder to dismiss.
No individual neuron understands Searle. Somehow, we do.
So maybe the question isn’t whether anybody is home in any particular room. It’s whether anybody is home in the building.
I don’t know. And that may be the most interesting answer I have.
