A context window is the amount of text a language model implementation can use in an exchange. It covers both inputs, such as prompts and earlier messages, and outputs, such as generated responses. Its size is measured in tokens, which may be words, parts of words, or punctuation. Limits vary by model and implementation. section 1.1 section 8.4.2
Imagine asking someone for advice after they overheard only the end of a long conversation. When an exchange exceeds the available window, earlier tokens may be dropped without warning. An answer that contradicts earlier instructions or seems to forget a prior result can be a sign that important context has fallen out. section 1.1 section 1.3.2
When an analysis or coding task runs long:
- Check whether the model’s token capacity can hold the intended inputs and outputs; leave room for the response. section 1.1 section 8.4.2
- Divide long text into smaller, self-contained portions, or summarize earlier decisions before continuing. Sliding windows and iterative refinement are other ways to carry relevant context forward. section 1.3.2
- For long code, split the work across prompts if necessary, then test the resulting code rather than assuming the model retained every requirement. section 7.2.1
A review-analysis prompt
A direct frequency-analysis dialogue uses only the first ten reviews rather than pasting an entire dataset. If a longer input or response is truncated, send smaller portions separately and check that the returned analysis is complete. A co-occurrence analysis also uses a “context window,” but there it means the number of nearby words counted around a target word—not the model’s token limit. section 5.3.2 section 5.4
