A context window is the maximum amount of text a language model can process in one request, measured in tokens. Tokens are small pieces of text; 8,192 tokens amount to roughly 6,000 English words. Think of the window as working space for the conversation, not permanent memory. section 11.4 section 11.1
A chatbot appears to remember because the app sends conversation history with each new message. The window must fit the user’s messages, the model’s earlier replies, and the reply being generated. After enough exchanges, that history may no longer fit. section 7.4.5 section 11.4
When a chat gets long
After twenty detailed exchanges, history could reach 5,000–10,000 tokens. If it exceeds the available window, the model may lose earlier messages or give confused, contradictory answers. Sending more context can also make each response slower. section 11.4 section 11.4.1
Table 11.1 Typical context window sizes for common local models
| Model |
Context window |
|---|---|
| gemma2:2b |
~8,192 tokens |
| llama3.2:3b |
~128,000 tokens |
| qwen2.5:3b |
~32,768 tokens |
Compare typical context-window capacities across local models. These figures describe model capacity, not necessarily the context Ollama has allocated; check the active context with ollama ps. section 11.4
Keep the full conversation visible to the user, but trim the messages sent to the model when necessary. Streamlit’s st.session_state can store the app’s messages; it does not increase the model’s context window. To inspect model details, including context length, use ollama show with the model name. section 11.9 section 11.1 section 4.5.3
