The model you're done improving

In February, Taalas showed a test chip fabbed on TSMC's 6nm process serving Meta's Llama 3.1 8B at 16,960 tokens a second — 48x an Nvidia GPU, 8.5x a Cerebras accelerator. Llama 3.1 shipped in mid-2024. The fastest inference silicon anyone has demonstrated was running a model that was already old when it ran it. AMD acquired Taalas on Thursday, announced at market close, terms undisclosed.

AMD did not buy speed. Speed was the demo. What AMD bought is the premise underneath it: that somewhere in the stack there is a model a company is finished improving. That premise has never held at the frontier. Where it does hold is the whole question.

Taalas chips don't keep weights in HBM. They etch them into the silicon — a mask-ROM recall fabric holding the weights, SRAM beside it for KV caches and fine-tuning adapters. Model-specific integrated circuits. The second-gen HC2 targets 20 billion parameters per chip, so fifty of them carry a trillion-parameter model through pipeline parallelism, against the few dozen GPUs and 2,000-plus Groq LPUs Nvidia's LPX racks need for the same work. The efficiency isn't in dispute. The commitment is.

Taalas's answer is that a re-spin isn't a restart: two layers of metal change, not the whole mask set. That's true, and it's the cheap kind of revision — metal-layer edits are how the industry has patched silicon for decades. But cost was never the binding constraint. The constraint is the decision. A metal spin still buys weeks of fab queue and a fresh validation pass, and it asks something no lab shipping monthly wants to do: name the version it has stopped working on.

Etching is a commitment device before it is a manufacturing process.

Which is why this ships, and not to the people buying racks. The likely shape is disaggregated — prompt processing on Instinct GPUs, token generation offloaded to Taalas parts — sold to model houses and inference providers, not enterprises. OpenAI, Anthropic, and Meta are already major Instinct customers, and Anthropic confirmed this week that it is hiring its own custom silicon team. The only companies that can safely freeze a model are the ones that control everything running on top of it.

And the freeze buys motion somewhere else. Cut cost per token by 10x and the obvious move is test-time scaling: let the model think longer, because thinking got cheap. The weights stop moving and the loop around them speeds up. Improvement doesn't stop — it migrates out of the model and into the scaffolding, which is where anyone running agents has watched it go for a year already.

The HC1 served a two-year-old model at 48x a GPU, and that was the point of the demo, not its apology. The bet isn't that models stop changing. It's that the part worth etching stops changing first.