chapter three

3 External knowledge and retrieval

 

This chapter covers

  • External knowledge as a context source beyond an LLM’s training data
  • Retrieval-augmented generation (RAG) to encode and query external knowledge
  • Knowledge graphs to structure and retrieve (graph RAG) external knowledge
  • Advanced retrieval approaches, including hybrid, agentic, and vectorless RAG
  • Cache-augmented generation (CAG) to preload external knowledge
  • Context stuffing for direct injection of external knowledge

Chapter 1 introduced external knowledge as one of the six context sources that LLM-based systems supply to the model. While an LLM’s parametric memory is fixed after training, external knowledge connects the model to factual, proprietary, or up-to-date information. Context engineering manages this connection through three primary strategies: retrieval, caching, and direct injection. Retrieval workflows, such as retrieval-augmented generation (RAG) and its advanced variants (graph-based, hybrid, agentic, and vectorless RAG), locate relevant passages dynamically at query time. Caching architectures, such as cache-augmented generation (CAG), preload reusable context to reduce redundant retrieval and latency. Finally, direct injection (often called context stuffing) loads complete documents directly into the context window. The following sections explore each approach, their trade-offs, and the criteria for choosing among them.

3.1 External knowledge

3.2 Retrieval-augmented generation

3.2.1 The indexing pipeline

3.2.2 The generation pipeline

3.2.3 Chunking strategies

3.2.4 Retrieval models and vector databases

3.2.5 RAG in production environments

3.2.6 RAG tools

3.3 Knowledge graphs

3.3.1 Graph structure and modeling

3.3.2 Graph RAG

3.4 Hybrid RAG

3.5 Agentic RAG

3.6 Vectorless RAG

3.7 Cache-augmented generation

3.8 Context stuffing

3.9 Choosing an approach

3.10 Hands-on

3.10.1 A minimal RAG implementation with an OpenAI model

3.10.2 RAG with an open-source generation model

3.10.3 Fully local RAG system

3.11 Summary