liveBook · Manning

Knowledge base

As explained in Domain-Specific Small Language Models

Retrieved outside information supplied to a model at inference time to ground its output.

A knowledge base supplies outside information to a language model when it answers a query. In retrieval-augmented generation (RAG), the system selects relevant material from a larger corpus and places it in the model’s context. The material may come from private or public sources; it need not be part of the model’s training data. section 2.2

A typical knowledge-base workflow has two phases:

  • Offline: extract documents, split their text into chunks, convert the chunks into embeddings, and store the embeddings with an index for search. Repeat this work when documents are added, removed, or updated. section 13.2
  • Online: embed the user’s query, find semantically similar chunks, and pass the retrieved text alongside the query to a generator model. section 13.2

This separates maintaining domain information from generating an answer. Retrieval is worth considering when domain-specific or unstructured data makes prompt engineering insufficient; fine-tuning is another option, and the choice depends on the use case. Retrieved context can support more accurate, up-to-date answers, while checking an output against its sources can improve transparency and accuracy. section 2.2

Querying a PDF

For a single-PDF knowledge base, extract page text with PyMuPDF, split it into sentence chunks, and embed the chunks with a SentenceTransformer model. Embed a query such as “blocked KV-cache,” rank chunks by similarity, and include the top passages in a prompt to a Phi-3 generator. This example assembles the steps with task-specific Python packages rather than a dedicated RAG framework. section 13.2

Figure 13.3 The generic workflow of a RAG system

The workflow diagram distinguishes preparing and indexing the corpus from retrieving context for each query. section 13.2

How Domain-Specific Small Language Models covers it

How deep it goes
Traces document preparation, embedding, retrieval, and generation through a single-PDF implementation.
Examples
Uses a PDF corpus and a “blocked KV-cache” query to demonstrate chunk retrieval and answer prompting.
What you can do after
Build it from scratch with task-specific Python packages for PDF extraction, embedding, retrieval, and generation.
How it connects
Connects knowledge bases to retrieval-augmented generation, embeddings, vector databases, prompt engineering, and fine-tuning.

Also explained in

Compare how 10 books explain knowledge base →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Domain-Specific Small Language Models. The section links point to the parts of the book it draws on.