Distillation
Concept
Training & Inference
Teaching a small model to imitate a big one.
You're looking at a concept explainer: what this term means in plain English, and the records it ties together.
Distillation trains a compact student on a larger teacher's outputs, keeping most capability at a fraction of the serving cost — the standard way 'mini' and 'flash' tiers are made. It is also a flashpoint: training on a rival API's outputs against its terms is industrial-scale imitation, an accusation that surfaced prominently around DeepSeek's rise.
How this connects to the rest of the stack
This record's links at a glance — click a group to jump to it.
Related
Sources
Distillation paper (Hinton et al.) ↗
Record reviewed as of 2026-01-15 — see Methodology.