Google TPU Pods ☆

Google scales TPUs in pods — 4,096 chips (v4), 8,960 (v5p), 9,216 (Ironwood) — wired by inter-chip interconnect in a 3D torus and reconfigured through optical circuit switches that route around failures without touching a cable. Gemini trains across multiple pods over the Jupiter network; the pod, not the server, is Google's hardware product.