liveBook · Manning

External memory

As explained in Retrieval Augmented Generation, The Foundational Ideas

Information stored outside a model's learned parameters in an external, searchable collection that can be updated independently.

External memory stores knowledge outside a model’s learned parameters in a structured, searchable corpus. A retriever gives the model access to that information on demand, like an open-book exam. section 6.1

Moving knowledge outside model parameters changes how it can be maintained and checked:

  • Add, delete, or modify indexed documents without retraining the model; this can keep information current and make domain specialization cheaper. section 6.1
  • Responses can cite source documents, but retrieval adds inference time and can fail to find relevant knowledge. section 6.1

Changing information

A financial service needs current market reports; a legal tool needs recent rulings; a clinical support system needs the latest medical research. section 6.1

Table 6.1 Parametric vs non-parametric knowledge
Feature
Parametric Knowledge (e.g., GPT-3)
Non-Parametric Knowledge (e.g., Atlas)
Knowledge Freshness
Static; frozen at the time of training.
Dynamic; updatable in near real-time.
Verifiability
Opaque; "black box" reasoning.
Transparent; responses grounded in citable sources.
Cost to Update
Extremely high (full model retraining).
Low (re-index external corpus).
Domain Specialization
Requires expensive fine-tuning or prompting.
Achieved by curating the external corpus.
Hallucination Risk
High when knowledge is outdated or missing.
Low; the model can state that knowledge is not found.
Failure Mode
Factual hallucination when knowledge is missing/outdated.
Retrieval failure; can return "knowledge not found."

Compare external memory with parametric knowledge across freshness, verifiability, update cost, specialization, and failure modes. section 6.1

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Defines external knowledge and compares its benefits and tradeoffs, but does not detail retrieval implementation.
Examples
Uses Wikipedia, company databases, market reports, court rulings, and medical research as examples of external knowledge.
What you can do after
Evaluate the freshness, verifiability, update cost, and retrieval tradeoffs of external knowledge.
How it connects
Contrasts non-parametric knowledge with parametric knowledge and connects external corpora to retrievers.

Also explained in

Compare how 2 books explain external memory →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.