Every accelerator in our database sits somewhere on one spectrum: flexibility versus specialization.
Why graphics chips, of all things
A CPU has tens of powerful cores optimized for doing different things fast. Graphics needed thousands of simple cores doing the same thing to every pixel — which happens to be exactly the shape of neural-network math. When AlexNet trained on two gaming cards in 2012, the match was proven, and NVIDIA — whose CUDA platform had already made GPUs programmable — owned the moment. Modern "GPUs" like the B200 barely do graphics: their tensor cores are matrix engines, their HBM feeds them, and the name is a fossil.
The TPU: same math, different philosophy
Google's TPUs strip away everything not needed for tensor math — no graphics heritage, simpler cores, a systolic-array design — and move the cleverness up a level: pods of thousands of chips with optical circuit switching act as one machine. TPUs are not sold; they are rented through Google Cloud, which is the point: the chip is a component of a vertically integrated system, tuned to Google's own frameworks (JAX, XLA).
Custom ASICs: hyperscaler economics
AWS Trainium, Microsoft Maia and Meta MTIA follow the TPU playbook: when you buy millions of chips a year for known workloads, designing your own (usually with Broadcom or Marvell) saves the margin NVIDIA would charge — often quoted as a 30–40% cost advantage per unit of work, at the price of software burden and less flexibility.
The far end: frozen silicon
Specialize further and you get inference-only designs — Groq's deterministic LPU, d-Matrix's in-memory compute — and at the extreme, Etched's Sohu, which physically hard-wires the transformer and can run nothing else. Maximum efficiency, maximum bet: if architectures shift, frozen silicon is scrap.
The pattern
Flexibility wins while algorithms churn; specialization wins when they settle. The industry's current answer is a portfolio: GPUs for the frontier and everything new, custom silicon for the stable expensive middle, specialists for high-volume inference niches. Watch where the architecture stops moving — that is where the next ASIC lands.
5
