Truncation manages a limited context window by retaining the newest messages that fit a token budget and discarding older content. It is fast, predictable, and requires neither additional LLM calls nor external services, making it suitable for transactional interactions such as password resets or self-contained FAQ questions. Its production risk is selective amnesia: the discarded messages may contain facts that remain important to the current response. section 4.7.2 section 4.8
Implement truncation as a deliberate baseline, then test whether conversation behavior requires stronger context management:
- Work backward from the latest message, accumulate tokens until the budget is reached, and drop everything older. section 4.7.2
- Prefer it when questions are largely independent and older conversation details do not affect safety or correctness. section 4.7.6
- Replace or supplement it with summarization when conversations exceed the budget but older flow still matters. section 4.7.6
- Add model-managed memory for critical facts and retrieval-augmented memory when users refer to previous sessions. section 4.7.6
When the boundary removes the wrong fact
A healthcare conversation may place a penicillin allergy in message 1, symptoms in message 5, and a preferred pharmacy in message 12. If truncation keeps only messages 31–50, all three facts disappear, allowing a later response to contradict earlier information or miss safety-critical context. section 4.7.2
Figure 4.7 Simple truncation drops older conversations from the context window. Messages 1-30 are dropped and only messages 31-50 are kept. Message 1 contains "I have a penicillin allergy", which is now lost, and model loses that information when answering the user’s question.

Use the figure to inspect how a fixed recent-message window removes earlier conversation facts. section 4.7.2


