A context window is the token capacity available to an LLM for the input and generated text of a request. In practice, context size includes the prompt, conversation history, retrieved material, and other input sent to the model. As conversations grow or prompts become more elaborate, context size can increase latency and cost while leaving less room for useful output. section 4.2.3 section 9.10.1
Treat context size as an engineering constraint:
- Measure tokens sent to the model alongside prompt-processing time, time to first token, total latency, and cost per request. section 4.2.3 section 4.6
- Trim or restructure overly long prompts when they add latency, increase evaluation cost, or crowd out important context. section 9.10.1
- Choose an evaluator with a context window large enough to see the evidence required for a long-form judgment. section 9.5.4
- Track context size separately from retrieval latency, because retrieved material can expand the request while retrieval creates its own bottleneck. section 4.2.3
Prompt debugging
A judge prompt that includes extensive instructions, source material, and evaluation history may become expensive and slow. Inspect its token usage, cap generated output where supported, and test missing, conflicting, or noisy context to confirm predictable failure behavior. section 9.10.1





































