Specialised Compute, Data Movement, and the Strange Return of the Coprocessor
A couple of weeks ago I wrote about an Intel 80287 floating-point coprocessor that I bought for my first PC around 1990. It was a separate piece of silicon designed to do something the main processor wasn’t particularly good at: floating-point mathematics.
Then I wrote Why the Memory Is Glued Down, which approached modern AI hardware from a different direction. That one was about distance. A language model consists of billions of numerical parameters, or weights. During inference, those numbers have to be available to the processor at extraordinary rates. This is why modern GPUs have high-bandwidth memory sitting extremely close to the processor rather than replaceable RAM sitting elsewhere in the machine.
Move some of a model from GPU memory into ordinary system RAM and you can experience the problem yourself. The model still works. It just gets much slower.
I ended that article thinking about the strange geometry of computing: moving the numbers closer and closer to the processor because moving data is increasingly one of the fundamental costs of AI.
It turns out there is another step. Don’t move the weights closer to the processor. Stop moving them at all.
Burn the model into the chip
Earlier this year, a Canadian startup called Taalas demonstrated something called the HC1. It runs Meta’s Llama 3.1 8B language model.
But saying it runs Llama is slightly misleading. On a conventional GPU, the model is software. Its billions of weights are stored in memory and repeatedly supplied to the processing hardware as tokens are generated.
Taalas has taken a radically different approach. The weights of the model are physically encoded into the silicon. Taalas calls this a Model-Specific Integrated Circuit, or MSIC.
Its HC1 technology demonstrator was manufactured using TSMC’s N6 process and contains 53 billion transistors on an 815 mm² die, sitting on a standard PCIe card drawing roughly 200 to 250 watts. Taalas reports that it can generate around 17,000 tokens per second per user running Llama 3.1 8B, with sub-millisecond latency to the first token. (Taalas)
Seventeen thousand. Not seventeen. Not seventeen hundred.
There are plenty of qualifications to attach to that number. It is a vendor benchmark, achieved with aggressive mixed 3-bit and 6-bit quantisation, on an 8-billion-parameter model rather than a current frontier model, and specialised hardware inevitably gives up flexibility in exchange for performance.
But the interesting thing isn’t really the benchmark. It is how they got there.
How the weights get burned in
The mechanism is more specific than “encoded into the silicon” suggests.
Taalas writes the trained weights directly into a mask-ROM-like fabric in the chip’s top metal and via layers, the same layers a foundry would normally use for ordinary interconnect. Only a small fraction of the mask set needs to change to burn in a different model, reportedly around two layers out of roughly a hundred. Everything else on the die is reusable.
The chip runs two separate memory fabrics side by side. A mask-ROM recall fabric holds the immutable base weights, fully digital, with each transistor apparently doing double duty: storing a few bits and performing the multiply that uses them, rather than storing a number and handing it off to a separate multiplier elsewhere on the die. An SRAM recall fabric handles the parts that are allowed to change while the model runs, principally the KV cache and any LoRA-style adapters.
There is no HBM stack, no CoWoS packaging, and the first-generation card doesn’t need liquid cooling. The weights never leave the die, so there is no per-token memory fetch to wait for. That is where the sub-millisecond first-token latency, and the headline throughput number, both come from.
The memory wall
A GPU is an extraordinarily fast calculator. The problem is keeping it supplied with numbers.
During autoregressive language-model inference, enormous quantities of model-weight data have to move between memory and the computational units. This is why the AI industry has spent so much money developing increasingly exotic memory systems.
DDR becomes GDDR. GDDR becomes HBM. Memory moves onto the graphics card. Then beside the processor. Interfaces become thousands of lanes wide. Packaging becomes increasingly sophisticated.
All of this is an attempt to solve the same underlying problem. The processor needs the numbers. So we keep bringing the numbers closer.
Taalas asks a wonderfully obvious question in retrospect: what if the numbers were already there?
Instead of storing the fixed model weights in HBM, its architecture incorporates them into a mask-ROM-like fabric on the chip itself. The things that genuinely need to change while the model is running, such as the KV cache and adapters, can remain in SRAM. (FutureParse)
The great weight-fetching operation largely disappears.
That’s memory moving towards the processor, taken to its logical extreme. The distance between the model and the computation approaches zero because the model becomes part of the computation.
Software becomes physical
This is the part I find stranger than the performance numbers.
We are accustomed to thinking about software and hardware as different things. My RTX 4080 doesn’t know what model it is going to run when NVIDIA manufactures it. Today it might run an image generator. Tomorrow it might run Llama. Then Blender. Then a game. That programmability is one of the GPU’s greatest strengths.
The model, meanwhile, is just data. I can download another one, delete it, quantise it or replace it.
Taalas deliberately gives up much of that freedom. The HC1 was made for Llama 3.1 8B. Want a fundamentally different base model and you need different silicon. The architecture retains some runtime flexibility, including support for adapters, but the fundamental model is fixed. (CNX Software)
Because so little of the mask set has to change between models, Taalas says a respin is a matter of weeks to a couple of months rather than the years associated with a full chip redesign. The next chip, HC2, is reportedly aimed at models around 20 billion parameters, with larger models handled by chaining several cards together rather than by redesigning the die again.
That sounds like an enormous disadvantage. And it is. Unless you don’t need to change the model. Then the disadvantage becomes the reason the thing can be so fast.
This is one of the oldest bargains in computing. Give up generality and gain efficiency.
The recurring pattern
Which brings us back to the underlying pattern rather than the specific chip that started it.
Floating-point maths was once specialised enough to deserve a separate chip. GPUs later specialised massively parallel computation. Eventually GPUs became programmable enough that we discovered they were useful for neural networks. AI accelerators then began specialising architectures around the particular mathematical operations used by neural networks.
And now we have reached something even more specific. We aren’t merely specialising the kind of mathematics the hardware performs. We can specialise the hardware around one particular collection of learned parameters.
The connection across four decades isn’t a direct engineering lineage. It’s a recurring pattern: generalise, find an important workload, specialise, integrate, then specialise again.
Forty years ago, floating-point mathematics was specialised enough to deserve its own chip. Today, apparently, an individual AI model can be.
The model on a card
There is something wonderfully physical about this.
Language models normally feel intangible. We talk about Llama, Qwen or GPT as though they are things, but ultimately a model is an enormous arrangement of numbers stored somewhere in a data centre or on a disk.
Taalas turns that abstraction back into an object. A particular trained model can become silicon. Put the hardware into a machine and the machine has access to that model at extraordinary speed.
It is tempting to call this the return of the add-on coprocessor card, though the resemblance only goes so far: those earlier chips were general-purpose accelerators, while model-specific silicon is committed to one specific set of trained weights. Even so, the resemblance is irresistible.
There’s something appealing about the idea of buying a card because you want a machine to contain a particular AI model, rather than downloading or loading one. Not download it. Not load it. Contain it.
Token abundance
There is another consequence that might matter much more than getting a chatbot to type impossibly fast.
Nobody needs to read 17,000 tokens per second. Machines do.
An AI agent might call a model repeatedly while completing one task: formulate a plan, call a tool, inspect the result, reconsider its approach, generate code, execute it, examine an error, modify the code, try again. More elaborate systems can run several approaches in parallel or use models to critique the outputs of other models.
Suddenly tokens are not primarily something displayed on a screen for a human to read. They become an internal computational resource. If inference becomes dramatically cheaper and faster, an AI system can afford to generate enormous quantities of intermediate work that nobody ever sees: ten attempts, a hundred, a thousand, most of them thrown away. The valuable output might still be three sentences.
This is sometimes described as token abundance: intelligence becoming cheap enough that software can spend enormous quantities of inference on ordinary problems.
Whether model-specific silicon ultimately provides that abundance remains to be seen. Taalas’s HC1 is a technology demonstrator, not proof that every future language model will be manufactured this way. General-purpose GPUs remain enormously valuable precisely because they can run different models, train new ones and perform many other workloads. Industry analysis currently points more towards specialised inference hardware coexisting with GPUs than replacing them. (TrendForce)
But AMD clearly thinks the idea is worth betting on. In August 2026, it agreed to acquire Taalas, a deal expected to close later this year. The strange little experiment of physically encoding a language model into silicon is now part of the strategy of one of the world’s major processor companies.
Not a brain, but something adjacent
So I would not write, anywhere in this piece, that Taalas works like the human brain. That is too strong a claim.
I would write something more interesting. It doesn’t make the chip a brain. But it removes one of the oldest architectural separations in computing: the split between where learned information is stored and where it is used.
And that raises a fascinating longer-term question.
If we can eventually combine model-specific, in-memory computation with locally modifiable weights, then we are no longer merely asking how to make GPUs faster.
We are asking whether the architecture of AI computing eventually moves away from the classic processor over here, memory over there arrangement altogether.
That is potentially a much bigger story than 17,000 tokens per second.
So
Specialised computation and the movement of memory towards the processor turn out to be heading towards the same place.
The GPU said: build enormous amounts of specialised parallel computation. HBM said: put the model’s numbers as close to that computation as physically possible. Model-specific silicon now says: why are we moving the fixed numbers at all?
Perhaps this turns out to be a niche architecture. Perhaps models change too quickly for fixed silicon to keep up. Perhaps some other approach wins. But the direction is fascinating.
For decades we have been moving memory and computation closer together. Eventually someone was bound to ask what happens if we stop treating them as separate things.
We’ve spent decades specialising the mathematics, then the parallel computation, then the memory. Now, apparently, we’re beginning to specialise the model itself.
The model isn’t running on the chip. The model is becoming the chip.
