A context window is the maximum number of tokens a language model can process in one interaction, counting both the input prompt and generated output. If a model supports 8,192 tokens and the input uses 200, only 7,992 remain for the response. Exceeding the limit can cause the request to be rejected. section 5.1.1
Because words and subwords are tokenized before processing, prompt length directly affects response capacity, latency, and cost. Budget system instructions, user content, retrieved context, and expected output together rather than treating the prompt as free space. section 5.1.1
Practical constraints shape application design:
- Provider limits vary: GPT-3 models are described with 4,096-token limits, while GPT-4 versions support 8,192 or 32,768 tokens. section 5.1.1
- Gemini 3 Pro and Gemini 3 Flash support up to 1,048,576 input tokens and 65,536 output tokens; other listed Gemini 2.5 variants support the same input limit. section 5.1.1
- Longer prompts can increase charges because services generally bill for total input and output tokens. section 5.1.1
- Agents must fit tool names, descriptions, arguments, and conversation content inside the model limit; many tools can therefore make an agent unusable. section 6.4.2
Count tokens before sending a prompt
import { encoding_for_model } from '@dqbd/tiktoken'; function countTokens(text, model = "gpt-3.5-turbo") { const enc = encoding_for_model(model); const tokens = enc.encode(text); enc.free(); return tokens.length; } const prompt = "Translate the following English text to French: 'Hello, how are you?'"; const systemMessage = "You are a helpful assistant that translates English to French."; const totalTokens = countTokens(prompt) + countTokens(systemMessage); console.log(`Total tokens: ${totalTokens}`); section 5.1.1
In a LangChain agent, reduce unnecessary tool context and computations when limits or latency become problems. Caching can also improve response efficiency when repeated work passes through retrievers, tools, and language models. section 6.4.2
