Context Window
Concept
Core Technical
The model's working memory — from 2K tokens to millions in five years.
You're looking at a concept explainer: what this term means in plain English, and the records it ties together.
The context window is how much a model can consider at once. GPT-3 held 2K tokens; Gemini holds a million; long-context serving is why the KV cache (the stored state for every token in context) dominates inference memory and why chips like Rubin CPX exist just for the 'prefill' of huge contexts. Long context competes with retrieval (RAG) as the way to give models knowledge.
How this connects to the rest of the stack
This record's links at a glance — click a group to jump to it.
Sources
Record reviewed as of 2026-01-15 — see Methodology.