Inference
Running the model — now the majority of AI compute and the focus of the buildout.
Anda sedang melihat entri penjelas konsep: apa arti istilah ini dalam bahasa yang mudah dipahami, dan entri-entri yang dikaitkannya.
Inference is every ChatGPT answer, Copilot completion and API call — and as products scaled, it overtook training as the dominant AI workload. It splits into prefill (reading the prompt, compute-bound) and decode (generating tokens, memory-bandwidth-bound), which is why HBM decides serving economics, why engines like vLLM matter, and why hardware now specializes by phase (Rubin CPX). 'Tokens per dollar per watt' is the metric the whole industry is converging on.
Bagaimana ini terhubung ke bagian lain dari stack
Tautan catatan ini sekilas — klik grup untuk langsung menuju ke sana.
Terkait
Sumber
Rekaman ditinjau per 2026-01-15 — lihat Metodologi.