liveBook · Manning

Context retention

As explained in AI for Everyday IT

Context retention is an AI model's ability to preserve, locate, and accurately recall information from earlier or distant parts of its input context.

Context retention is an AI model’s ability to hold and use information during a conversation or series of tasks. It differs from statefulness: statefulness maintains conversation flow, while context retention determines how much information remains available and for how long. Strong retention supports relevant, coherent replies; weak retention can produce forgotten details, off-topic answers, or hallucinations. section 3.1.3 section 3.6

Use these practical signals and safeguards when working with long AI conversations:

  • Treat each chat session as an isolated context; information from another chat generally cannot be referenced. section 3.1.3
  • Restate key points or provide short summaries when a discussion becomes complex, helping the model stay focused. section 3.1.3
  • Expect recall quality to decline as token usage grows, especially when specific details are buried in large amounts of text. section 3.2.1

A conversation that outgrows its context

A user kept one ChatGPT conversation open for two months. As the history grew beyond what the model could effectively process, its answers deteriorated: it made false claims about creating documents in the background and promised future notifications. This illustrates why a long-running chat should be summarized or restarted rather than treated as unlimited memory. section 3.1.3

Table 3.1 Examples of token limits in common AI models
Model
Token limit (input)
Token limit (output)
Approximate word count
OpenAI GPT-3.5-Turbo
4K
4K
3K total
Microsoft Copilot
4K–8K
4K–8K
3K–6K total
OpenAI GPT-4
8K
8K
6K total
OpenAI GPT-4o
128K
16K
96K input, 12K output
OpenAI o1
128K
32K
96K input, 24K output
OpenAI o3-mini
200K
100K
150K input, 75K output
Anthropic Claude 2 and 3
100K–200K
100K–200K
75K–150K total
Google Gemini 2 Flash
1M
32K
750K input, 24K output
Google Gemini 2 Pro
Up to 2M
64K
750K input, 48K output

Compare listed input and output token limits across GPT, Copilot, Claude, and Gemini models to estimate how much conversation or source material a system can handle. section 3.2.1

How AI for Everyday IT covers it

How deep it goes
Explains memory duration, session boundaries, token-related decline, and practical summaries without covering model internals.
Examples
Uses isolated chat sessions, a two-month conversation, summaries, and needle-in-a-haystack retention tests.
What it assumes
Builds from zero by defining context, tokens, statefulness, and token limits without requiring programming or AI expertise.
What you can do after
Apply summaries and restatements to preserve conversational focus, and evaluate recall with buried-detail tests.
How it connects
Connects context retention with statefulness, coherence, prompts, tokens, token limits, and hallucination.
Cover of AI for Everyday IT
liveBookmanning.com
Read AI for Everyday IT on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI for Everyday IT. The section links point to the parts of the book it draws on.