Inference
Running the model — now the majority of AI compute and the focus of the buildout.
Stai visualizzando una scheda concettuale: cosa significa questo termine in linguaggio chiaro, e i record che mette in relazione.
Inference is every ChatGPT answer, Copilot completion and API call — and as products scaled, it overtook training as the dominant AI workload. It splits into prefill (reading the prompt, compute-bound) and decode (generating tokens, memory-bandwidth-bound), which is why HBM decides serving economics, why engines like vLLM matter, and why hardware now specializes by phase (Rubin CPX). 'Tokens per dollar per watt' is the metric the whole industry is converging on.
Come si collega al resto dello stack
I collegamenti di questa scheda a colpo d'occhio — clicca un gruppo per raggiungerlo.
Correlati
Fonti
Record revisionato al 2026-01-15 — vedi Metodologia.