TensorRT-LLM & Dynamo
Software
Inference
Active
NVIDIA's tuned inference stack — peak tokens from its own silicon.
You're looking at a software record: a tool or platform in the AI stack — who maintains it and what hardware it runs on.
TensorRT-LLM compiles models into aggressively optimized NVIDIA kernels (FP8/FP4, speculative decoding, in-flight batching); Dynamo (2025) orchestrates disaggregated prefill/decode serving across racks. Together they are how NVIDIA converts hardware advantage into benchmark-leading tokens per second.
How this connects to the rest of the stack
This record's links at a glance — click a group to jump to it.
Runs on
Maintained by
Competes with
Sources
Record reviewed as of 2026-01-15 — see Methodology.