{"data":{"slug":"inference","etype":"concept","name":"Inference","cat":"Training & Inference","subtitle":"Running the model — now the majority of AI compute and the focus of the buildout.","summary":"Inference is every ChatGPT answer, Copilot completion and API call — and as products scaled, it overtook training as the dominant AI workload. It splits into prefill (reading the prompt, compute-bound) and decode (generating tokens, memory-bandwidth-bound), which is why HBM decides serving economics, why engines like vLLM matter, and why hardware now specializes by phase (Rubin CPX). 'Tokens per dollar per watt' is the metric the whole industry is converging on.","attrs":null,"kf":null,"aliases":["serving","prefill","decode"],"ticker":null,"exchange":null,"hq":null,"founded":null,"website":null,"status":null,"region":null,"asof":"2026-01-15","sources":[{"url":"https://arxiv.org/abs/2309.06180","label":"vLLM paper"}]},"meta":{"found":true,"count":1,"served_from":"site cache (no live upstream calls)","as_of":"2026-10-14T05:44:24+00:00","disclaimer":"Research and information only — not investment advice. Structured records carry basis labels (vendor spec / reported / estimate); market quotes are delayed/indicative.","attribution":"Records compiled from vendor documentation, official filings and reputable public reporting; market quotes via financial data providers. See /data-sources."}}