High-Bandwidth Memory (HBM) ☆

HBM stacks DRAM dies vertically, connects them with thousands of through-silicon vias, and mounts the stack millimeters from the processor on a silicon interposer. The result is terabytes per second of bandwidth that ordinary DIMMs cannot approach — and because model inference is mostly memory-bound, HBM capacity and bandwidth, not FLOPS, usually decide real AI performance. Only SK hynix, Samsung and Micron make it, and supply has been sold out roughly a year ahead throughout the boom.