Step-back prompting improves retrieval when a detailed question matches narrow document chunks but misses the broader context needed for a complete answer. It reformulates the question into a more general version, retrieves that higher-level context, and combines it with the original retrieval context. section 9.3 section 9.0
A typical workflow retrieves detailed context for the user question, generates and executes an abstract question, then supplies the original question and both contexts to the answering LLM. For example, a Brighton trip question can become a question about planning a successful trip to a coastal city. section 9.3 section 9.3.1
Build the technique as a small question-generation chain followed by ordinary RAG:
- Prompt the LLM to generate a less specific question that can retrieve wider context. section 9.3
- Pipe the prompt through the model and output parser, then send the generated question to the vector-store retriever. section 9.3.1
- Combine retrieved context with the original question and instruct the answering model to admit when it does not know. section 9.3.2
Figure 9.4 Step-back question workflow
The workflow diagram clarifies how detailed retrieval, abstract-question generation, broader retrieval, and final synthesis fit together. section 9.3
From a specific question to broader retrieval
For “Can you give me some tips for a trip to Brighton?”, generate “What are some general tips for planning a successful trip to a coastal city?” Retrieve the broader context, then produce the Brighton-specific answer using that context.
Step-back prompting broadens the question rather than changing document chunking or embeddings. Model choice affects abstraction quality: GPT-5 is preferred in the example over smaller variants, but experimentation remains necessary because retrieval performance depends on the dataset. section 9.3.1 section 9.6
