Quantization
Concept
Training & Inference
Fewer bits per weight: how models shrink to fit real hardware.
You're looking at a concept explainer: what this term means in plain English, and the records it ties together.
Quantization stores weights and activations in fewer bits — 8, 4, even fewer — trading a sliver of accuracy for big memory and speed gains. It is why a 70B model runs on a laptop (GGUF 4-bit), and why hardware generations chase ever-lower precisions: Hopper's FP8 and Blackwell's FP4 are quantization cast in silicon.
How this connects to the rest of the stack
This record's links at a glance — click a group to jump to it.
Sources
Record reviewed as of 2026-01-15 — see Methodology.