liveBook · Manning

Lost in the middle

As explained in Vibe Engineering

Lost in the middle is a long-context failure pattern in which a language model retrieves information near the beginning or end more reliably than information placed in the middle.

A model can receive the right information and still fail to use it. In long inputs, details near the beginning or end are often retrieved more reliably than details buried in the middle. This positional effect is called lost in the middle and can produce a U-shaped performance curve. section 3.4.1

Repository-sized context

When a model was given smaller slices of source code, around 20,000–30,000 tokens, it identified dependencies and suggested fixes reasonably well. As the prompt grew into the hundreds of thousands of tokens, it hallucinated functions, misremembered file structures, and ignored instructions buried in the middle. section 3.4.1

Lost in the middle is distinct from context rot: accuracy can decline as total context grows even when the relevant information is favorably placed. The effects can compound. A production rule of thumb puts noticeable degradation around 40% window utilization, but that is a heuristic, not a universal threshold. Treat the advertised window as headroom rather than a target to fill. section 3.4.1

Avoid overloading the prompt. Retrieval-augmented generation (RAG) searches an indexed repository and adds relevant chunks to the question instead of sending thousands of unrelated lines. For an authentication fix, that might mean retrieving the login handler and its tests. Relevant chunks can also be placed where the model is more likely to attend to them. section 3.5.1

Figure 3.7 Context-aware assistance flow. The developer's query is converted into an embedding, matched against the indexed codebase, and the most relevant chunks are injected into the prompt before the model generates its response.

Follow the query-to-response flow: a query is matched against an indexed codebase, and relevant chunks are added to the prompt. section 3.5.1

When automatic retrieval repeatedly brings in the wrong material, scope the context manually: pin files, mention a directory, exclude paths, or start a clean session. Context engineering includes choosing and ordering information, not just phrasing the question. section 3.5.1 section 3.0

How Vibe Engineering covers it

How deep it goes
Explains the U-shaped positional effect, contrasts it with length-related degradation, and gives a utilization heuristic.
Examples
Uses expanding source-code prompts and an authentication handler with related tests to illustrate failure and selective retrieval.
What you can do after
Diagnose/debug with it by recognizing buried instructions as a possible cause of incorrect agent output.
How it connects
Connects context engineering with retrieval-augmented generation and context rot.

Also explained in

Compare how 3 books explain lost in the middle →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Vibe Engineering. The section links point to the parts of the book it draws on.