A context window is an LLM's finite working space for tokenized input and generated output. Its size constrains how much prompt, retrieved information, conversation history, and response content can fit into one interaction. section 12.1.1 section 12.2.2
Treat the window as an architectural budget rather than a headline model capability. Retrieval volume, chunk length, system instructions, user queries, and max_tokens compete for space. Exceeding the input limit fails the request; exhausting the output allocation can truncate a response mid-sentence. section 12.1.1
Plan context budgets by:
- Limiting how many document chunks retrieval supplies and controlling their length. section 12.1.1
- Reserving space for system instructions, the user query, and a complete generated response. section 12.1.1
- Splitting complex work into focused, composable steps instead of sending an entire documentation corpus in one prompt. section 12.2.2
A retrieval budget
Fifty retrieved chunks at 500 tokens each consume 25,000 tokens. Adding a 2,000-token system prompt and a 100-token query uses 27,100 tokens, leaving roughly 5,000 tokens in a 32,000-token window for generation. section 12.1.1
