Lost in the middle
Lost in the middle is the tendency of a language model to use relevant information in the middle of a long context less effectively than information near its beginning or end.
Vibe Engineering
Lost in the middle is a long-context failure pattern in which a language model retrieves information near the beginning or end more reliably than information placed in the middle.
- How it is used
- Identifies a context failure in which useful information is supplied but poorly positioned for retrieval.
- How deep it goes
- Explains the U-shaped positional effect, contrasts it with length-related degradation, and gives a utilization heuristic.
- Examples
- Uses expanding source-code prompts and an authentication handler with related tests to illustrate failure and selective retrieval.
- What you can do after
- Diagnose/debug with it by recognizing buried instructions as a possible cause of incorrect agent output.
Retrieval Augmented Generation, The Foundational Ideas
A language-model performance degradation caused by relevant information being placed amid distracting or competing context.
- How it is used
- Diagnoses why retrieval systems and long-context prompts overlook relevant evidence buried among competing documents.
- How deep it goes
- Explains positional neglect, retrieval noise, quadratic attention costs, and independent encoding, but does not detail training procedures.
- Examples
- Uses mashed-together chapters, irrelevant retrieved documents, and 1,000-document queries to illustrate positional and computational failures.
- What you can do after
- Diagnose/debug with it by recognizing when retrieval noise or concatenated long contexts obscure evidence, then compare independent encoding with prompt stuffing.
Read how Retrieval Augmented Generation, The Foundational Ideas explains lost in the middle →
Designing AI Systems
The lost in the middle phenomenon is a language-model behavior in which information positioned in the middle of a long context receives less attention than information near the beginning or end.
- How it is used
- Guides context assembly by placing high-value information at the beginning or end of a model input.
- How deep it goes
- Explains the observed attention pattern and translates it into ordering and budget decisions, without detailing model internals.
- Examples
- Uses structured memories, rolling summaries, recent verbatim messages, and document-versus-history allocation examples.
- What you can do after
- Apply it via context assembly by ordering sources and tuning their token-budget shares for the application.
Also covered in
- Context EngineeringIn early access (MEAP)
- AI Agents in Action, Second Edition
- LLM Customization and Fine-TuningIn early access (MEAP)
- Build an AI Agent (From Scratch)
What to know first
- context windowA context window is a bounded span of sequence elements or contextual information that a model or processing operation can consider together.
- promptA prompt is text or other input supplied to an AI model to provide instructions, context, or other information that guides its response or generated output.
- hallucinationA hallucination is model-generated content that is false, fabricated, or unsupported by relevant evidence but presented as plausible or factual.
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
Related concepts
- attention budgetproduced by · An attention budget is the finite capacity available to an agent or model to attend to information or events while performing a task.
- text chunkaddressed by · A text chunk is a bounded segment of a larger text treated as a unit for processing, storage, or retrieval.
- context failurekind of · Context failure is an insufficiency or mismatch in contextual information that prevents a system from adequately interpreting intent, assessing relevance, or performing a task.
- demonstrationcontrasts with · A demonstration is an example, often pairing an input with an output, that illustrates a task or desired behavior for training or guiding a model.
- context assemblyconfigures · Context assembly is the process of selecting, organizing, and combining relevant information and instructions into context for a model or other downstream processing.
- context engineeringused by · Context engineering is the deliberate selection, organization, and management of information supplied to an AI system to support effective task performance.
Questions about lost in the middle
What is lost in the middle?
Lost in the middle is the tendency of a language model to use relevant information in the middle of a long context less effectively than information near its beginning or end.
Which Manning books explain lost in the middle?
Vibe Engineering, Retrieval Augmented Generation, The Foundational Ideas and Designing AI Systems.
What should I know before learning lost in the middle?
Manning books that teach lost in the middle usually introduce context window, prompt, hallucination and language model first.
What is the difference between lost in the middle and demonstration?
Lost in the middle is the tendency of a language model to use relevant information in the middle of a long context less effectively than information near its beginning or end. A demonstration is an example, often pairing an input with an output, that illustrates a task or desired behavior for training or guiding a model.
What is lost in the middle a kind of?
Lost in the middle is a kind of context failure. Context failure is an insufficiency or mismatch in contextual information that prevents a system from adequately interpreting intent, assessing relevance, or performing a task.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.


