Floating Point, Specialised Compute, and the Persistence of a Good Idea
I am staring at the old 80287 board from my first PC. I have kept it for decades, stuck to the chassis of successive computers. Today it sits beside a workstation with an NVIDIA RTX 4080.
That got me thinking about floating point.
When I installed the 287 around 1990, floating-point maths was still specialist enough to justify a separate chip. The 286 could do general computing perfectly well without it. The 287 existed for software that needed more serious numerical work, such as CAD, scientific applications and rendering.
I bought the chip myself and installed it into the empty socket on the motherboard. It was my first customisation. I remember it being relatively cheap and I thought it would improve Flight Simulator 4. (It didn’t) Plus I wanted a higher-spec PC.
The main market for the Floating Point Numbers (FPU) was not gaming. It was business, scientific and technical software: CAD packages like AutoCAD, 3D modelling tools like 3D Studio, and ray-tracers like POV-Ray. Games mostly ignored it. Wolfenstein 3D, released in 1992, used fixed-point arithmetic and lookup tables rather than relying on floating-point hardware, a sensible choice when many target PCs either lacked an FPU or could not perform floating point cheaply enough for a fast action game. Doom’s source code, two years later, still explicitly defines its core arithmetic as 32-bit 16.16 fixed-point. Some flight simulators of the era did make use of a coprocessor where one was present, but this was the exception, not the rule.
Quake, in 1996, is where the story turns, and the overlap is what makes it interesting. Its original renderer was still software, running on the CPU with heavily optimised C and assembly, but it made substantial use of the CPU’s floating-point hardware to do it. This was exactly the period when consumer 3D acceleration was beginning to take off. The FPU had become genuinely useful to mainstream real-time 3D at the very moment a new kind of hardware was arriving to take the workload somewhere else entirely.
Intel had already integrated the FPU into the 486DX, folding a capability that once required a separate chip onto a single die. That is simply what semiconductor integration tends to do over time: functions that once needed multiple packages get absorbed into one as the process allows it.
Dedicated 3D graphics hardware arrived, driven largely by games, and evolved into the programmable GPU. By the GeForce 6 generation, programmable shader operations ran in FP32. By the GeForce 8800 era, NVIDIA was describing its architecture as a large grid of data-parallel floating-point processors. When NVIDIA opened that architecture up for general-purpose computing with CUDA in 2006, and researchers began training neural networks on GPUs around 2012, the GPU’s second life began. That capability turned out to be useful for something else entirely: artificial intelligence.
A modern language model ultimately represents information numerically. Tokens become vectors. Those vectors pass through layers of matrix operations, using floating-point formats such as FP32, FP16, BF16 and increasingly FP8. What appears at the top as language, images or reasoning emerges from an enormous amount of floating-point arithmetic underneath.
It is worth putting a number on that gap. My 80287 could manage floating-point maths at roughly a few hundred thousand operations per second at best. A modest seven-billion-parameter language model needs something on the order of 14 billion operations to generate a single token. As a deliberately ridiculous comparison: one 287 would need tens of hours to work through the arithmetic behind a single token. To reach even one token per second, you would need something like a hundred thousand of them, setting aside the small matter that no such system could actually hold the model’s weights, move data between chips fast enough, or be wired together in any coherent way at all.
The 80287 did not become the GPU. There is no direct engineering lineage between them. What connects them is a pattern, not a part: specialise, integrate, then specialise again once the workload outgrows what general-purpose hardware can do.
It would be tempting to read the x87 coprocessor’s eventual decline as evidence that floating point itself had reached a dead end. The opposite happened. Floating-point hardware moved into the CPU, while graphics created demand for vastly more numerical computation than a CPU alone could efficiently provide. As graphics processors became programmable, massively parallel floating-point computation became commercially abundant.
Then we discovered new things for all that arithmetic to represent.
The 287 was not backing a dead end. It was early to a party that had not started yet. Decades later, graphics built the venue, GPUs filled it with parallel maths, and AI found an entirely new reason to turn up.
