A knowledge base supplies outside information to a language model when it answers a query. In retrieval-augmented generation (RAG), the system selects relevant material from a larger corpus and places it in the model’s context. The material may come from private or public sources; it need not be part of the model’s training data. section 2.2
A typical knowledge-base workflow has two phases:
- Offline: extract documents, split their text into chunks, convert the chunks into embeddings, and store the embeddings with an index for search. Repeat this work when documents are added, removed, or updated. section 13.2
- Online: embed the user’s query, find semantically similar chunks, and pass the retrieved text alongside the query to a generator model. section 13.2
This separates maintaining domain information from generating an answer. Retrieval is worth considering when domain-specific or unstructured data makes prompt engineering insufficient; fine-tuning is another option, and the choice depends on the use case. Retrieved context can support more accurate, up-to-date answers, while checking an output against its sources can improve transparency and accuracy. section 2.2
Querying a PDF
For a single-PDF knowledge base, extract page text with PyMuPDF, split it into sentence chunks, and embed the chunks with a SentenceTransformer model. Embed a query such as “blocked KV-cache,” rank chunks by similarity, and include the top passages in a prompt to a Phi-3 generator. This example assembles the steps with task-specific Python packages rather than a dedicated RAG framework. section 13.2
Figure 13.3 The generic workflow of a RAG system
The workflow diagram distinguishes preparing and indexing the corpus from retrieving context for each query. section 13.2









