liveBook · Manning

Parametric knowledge

As explained in LLM Evaluation and Alignment, The Foundational Ideas

Parametric knowledge is information encoded in a model’s learned parameters rather than retrieved from an external source at inference time.

Parametric knowledge is information, facts, and patterns learned during training and stored in a neural network’s weights and parameters. It contrasts with context memory, which is supplied during inference. In production systems such as RAG, retrieved documents provide grounding, while additional claims may come from parametric knowledge. section 5.1.1

Use the distinction when evaluating generated claims:

  • Separate source-grounded claims from claims generated from the model’s learned parameters. section 5.1.1
  • Route source-based claims to reference verification and open-domain claims to consistency-based checks such as SelfCheckGPT. section 5.1.1
  • Treat repeated agreement as evidence of stable learned knowledge, not proof of factual correctness. section 5.2.1

Stable and unstable recall

Repeatedly asking for the author of the 2020 paper “GLU Variants Improve Transformer” produces the stable answer Noam Shazeer. Asking about a less reliably known sparse-mixture-of-experts paper may produce different co-author lists across generations, suggesting plausible construction rather than stable parametric knowledge. section 5.2.1

Figure 5.2 A production hallucination detection pipeline. Generated responses are decomposed into claims, routed to the appropriate verifier, aggregated into response-level metrics, and converted into actions such as logging, review, regeneration, or blocking.

Trace how generated responses are decomposed into claims, routed to source-based or consistency-based verifiers, aggregated into faithfulness metrics, and linked to production actions. section 5.1.1

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Defines storage in model parameters and explains consistency-based detection, but does not descend into parameter-level retrieval internals.
Examples
Uses repeated paper-authorship questions to contrast stable factual recall with variable hallucinated details.
What it assumes
Contrasts learned parameter storage with inference-time context and assumes no specialized model-internals background.
What you can do after
Evaluate generated claims by routing grounded content to reference checks and ungrounded content to consistency-based checks.
How it connects
Connects parametric knowledge with context memory, RAG, open-domain tasks, closed-domain tasks, and SelfCheckGPT.

Also explained in

Compare how 3 books explain parametric knowledge →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.