Transformer ☆

The transformer processes every token in a sequence in parallel, using attention to let each token consult every other. That parallelism is why GPUs and transformers co-evolved: the architecture is essentially a giant matrix-multiplication workload, which is exactly what accelerator silicon does best. GPT, Claude, Gemini, Llama, image diffusion backbones and even robot policies are transformers; challengers (state-space models, hybrids) so far nibble at its edges. One paper — 'Attention Is All You Need' — sits under roughly a trillion dollars of infrastructure.