Context window
In shortThe context window is the maximum amount of text, counted in tokens, that a model can read and write in one request.
In plain words
The context window is the model's working memory for one request. Anthropic describes it as all the text a model can reference when it writes a response, including the response itself. It is not the model's training data. Everything you send counts: the system prompt, every earlier message, pasted files, tool results and your tool definitions. The answer being written counts too, and so do any thinking tokens.
Sizes depend on the model. Anthropic lists windows of up to 1 million tokens for some of its models and notes that a larger window is not automatically better: as the token count grows, accuracy and recall degrade, an effect it calls context rot. What you put in matters as much as how much fits.
Why it matters
- Cost grows with every turn. An API does not remember a chat. Each turn resends the whole history as input, and you pay for those tokens again. Prompt caching can lower that bill.
- Quality can drop. A very long input can bury the detail you need.
- Hard stops. If your input alone is larger than the window, Anthropic's API rejects the request with an error. Chat apps may instead drop the oldest messages.
- Alternatives. RAG sends only the relevant passages instead of whole documents.
Example
You paste a long report into a chat and ask 30 questions in a row. Every question carries the report, your earlier questions and all earlier answers. By question 30 the request is large and costly, and the model may miss a detail from the middle of the report. Better: start a fresh chat for a new topic, paste only the needed sections, or let a search step pick the passages.
Common misconceptions
- "A big window means the model remembers me." The window covers one request. A model remembers across chats only if the app stores your notes and sends them again.
- "Context window and training data are the same." They are not. Training data shaped the model earlier; the window holds what you give it now.
- "Output does not count." It does. Input and output share the same window.
Questions and answers
What happens when the context window is full?
With an API, a request whose input exceeds the window is rejected with an error (Anthropic returns a 400 error). Chat apps may drop the oldest messages or summarise them to make room.
Do output tokens count toward the context window?
Yes. The answer the model writes, including thinking tokens, counts toward the same window as your input.
Is a bigger context window always better?
No. Accuracy and recall can degrade as the input grows. Cost also rises, because every token is billed. Send what the task needs.
Read next
- Token
A token is a small chunk of text, often a word piece, that an AI model reads, writes and bills.
- Prompt caching
Prompt caching lets an AI provider reuse the already processed start of your prompt, so repeated parts are cheaper and faster.
- RAG (retrieval-augmented generation)
RAG (retrieval-augmented generation) first looks up relevant passages in your documents, then gives them to the model so it answers from them.
- Large language model (LLM)
A large language model (LLM) is an AI system trained on huge amounts of text to predict the next token, which lets it write, summarise and answer.
- Token estimator: roughly count tokens in your text
A free tool that estimates how many tokens a text holds. It is a rough guide, not an exact count.