liveBook · Manning

Non-parametric approach

As explained in Retrieval Augmented Generation, The Foundational Ideas

A non-parametric approach stores task knowledge in an external information source rather than exclusively in learned model parameters.

A non-parametric approach separates a model’s reasoning engine from an external, updatable knowledge base. Updating stored knowledge can then avoid expensive model retraining. section 6.0

In retrieval-augmented generation, the retriever finds relevant documents and the reader uses them to generate an answer. Joint training lets reader performance directly teach the retriever which documents support correct answers. section 6.0

This approach is useful when knowledge changes or training data is scarce:

  • Keep knowledge current through an external document index rather than repeatedly retraining model parameters. section 6.0
  • Use aligned retrieval and reading to support few-shot performance with smaller models. section 6.0
  • Adapt query behavior through query-side fine-tuning without re-indexing the entire corpus. section 6.0

Atlas

Atlas applies this design to few-shot retrieval-augmented language modeling, combining an external knowledge source with jointly trained retrieval and reading components. section 6.0

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains decoupling, joint retriever-reader training, and query-side updates, but stops short of implementation details.
Examples
Uses Atlas, few-shot learning, document indexes, and query-side fine-tuning as concrete examples.
What you can do after
Apply it via retrieval-augmented systems that update knowledge efficiently and improve retrieval using reader feedback.
How it connects
Connects external knowledge bases with retrieval-augmented generation, retrievers, readers, and model parameters.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.