Retrieval-augmented generation (RAG) lets an application answer questions about information outside an LLM’s training data, including confidential documents and events after its training cutoff. It retrieves relevant document content and supplies it as prompt context. section 1.3 section 4.0
Instead of stuffing entire rulebooks into a prompt, RAG splits documents into chunks, embeds them, stores them in a vector store, and retrieves chunks similar to the user’s question. The application then adds those chunks to the prompt. section 4.1 section 4.7
A Spring AI implementation typically proceeds as follows:
- Add a vector-store starter, such as the Qdrant integration, and ensure the store contains loaded document chunks. section 4.4 section 4.1
- Inject a VectorStore and create a SearchRequest using the user’s question as the query. section 4.4.1
- Filter results with metadata such as a normalized game title when one vector store contains documents for multiple subjects. section 4.4.1
- Control retrieved context with Top-K and similarity thresholds, balancing relevance against prompt tokens and request cost. section 4.4.1
- Insert the retrieved text into a prompt that requires the model to answer from the supplied documents and say “I don’t know” otherwise. section 4.4.2
Game rules assistant
A Board Game Buddy service retrieves rule chunks for the requested game before calling the model. A question about Burger Battle’s Grave Digger card can receive an answer from loaded rules; a question about Carcassonne returns “I don’t know” until Carcassonne’s rules are loaded. section 4.4.2
Figure 4.1 RAG involves finding documents relevant to the question and adding them as context in the prompt.
Use the diagram to trace the flow from document loading and vector storage through similarity retrieval to prompt context. section 4.1
