Training a frontier model is a conversation: every few milliseconds, thousands of GPUs must agree on gradient updates. The network that carries that conversation decides how much of your billion-dollar cluster you actually get to use.
Two networks, two physics
Scale-up is the short-range, memory-speed domain: NVLink joins 8 GPUs on a board — or 72 in a NVL72 rack — at up to 1.8 TB/s each, close enough that they can share each other's memory. This domain is NVIDIA's deepest moat; the industry's answer is the open UALink standard, arriving with AMD's Helios racks.
Scale-out connects racks into clusters: 400–800 Gb/s per GPU over InfiniBand or Ethernet, in fat-tree or rail topologies designed so any GPU can reach any other in a few hops. Collective operations (NCCL's all-reduce) synchronize the fleet; a single slow link drags everyone.
The InfiniBand-vs-Ethernet war
InfiniBand (NVIDIA's, via Mellanox) owned big training for a decade: lossless, low-jitter, with in-network reduction. But hyperscalers hate single-vendor dependencies, and Ethernet's ecosystem (Broadcom switch silicon, Arista systems) is vast and cheap. The Ultra Ethernet Consortium retrofitted Ethernet with the missing manners (modern RDMA, packet spraying), NVIDIA hedged with its own Spectrum-X — which runs Colossus — and by 2025 the momentum question was settled enough that even NVIDIA sells both.
Optics: the light bill
Between racks, copper gives out; every port terminates in an optical transceiver — a cluster consumes them by the hundred thousand, at hundreds of dollars and several watts each. That line item built the fortunes of Coherent, Lumentum, Fabrinet and Credo, and drives the next transition: co-packaged optics, moving lasers onto the switch package to cut the power ~3x.
Why this layer decides winners
Cluster utilization — the fraction of theoretical FLOPS you actually harvest — lives or dies on the fabric. Meta's Llama team disclosed heroic networking engineering; xAI's Ethernet bet proved at 100k scale; and every 2026-era rack architecture (NVL576, Helios) is really a networking claim wearing a GPU costume.
6
