liveBook · Manning

T5-v1.1

As explained in Retrieval Augmented Generation, The Foundational Ideas

T5-v1.1 is a T5 model variant pretrained only on unlabeled text rather than supervised datasets.

T5-v1.1 is a sequence-to-sequence language model variant used as Atlas’s reader. Unlike the original T5, it is pretrained only on unlabeled text, avoiding supervised-dataset contamination that could overlap with downstream evaluation tasks. section 6.1

Within retrieval-augmented inference, T5-v1.1 provides the reader stage:

  • Receives documents selected by a dual-encoder retriever. section 6.1
  • Generates responses as part of a modular system connected to an external knowledge base. section 6.1
  • Supports more credible few-shot evaluation by excluding supervised pretraining data that might overlap with evaluation tasks. section 6.1
Table 6.1 Parametric vs non-parametric knowledge
Feature
Parametric Knowledge (e.g., GPT-3)
Non-Parametric Knowledge (e.g., Atlas)
Knowledge Freshness
Static; frozen at the time of training.
Dynamic; updatable in near real-time.
Verifiability
Opaque; "black box" reasoning.
Transparent; responses grounded in citable sources.
Cost to Update
Extremely high (full model retraining).
Low (re-index external corpus).
Domain Specialization
Requires expensive fine-tuning or prompting.
Achieved by curating the external corpus.
Hallucination Risk
High when knowledge is outdated or missing.
Low; the model can state that knowledge is not found.
Failure Mode
Factual hallucination when knowledge is missing/outdated.
Retrieval failure; can return "knowledge not found."

Compare T5-v1.1’s reader role with the retriever and the broader tradeoffs between frozen parametric knowledge and updateable non-parametric knowledge. section 6.1

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains its reader role and pretraining distinction, but does not detail internal layers, optimization, or generation procedures.
Examples
Uses Atlas’s reader, external knowledge base, and few-shot evaluation contamination as concrete examples.
What it assumes
Assumes familiarity with retrieval augmentation and sequence-to-sequence readers while distinguishing supervised from unlabeled pretraining.
What you can do after
Evaluate how unlabeled-only pretraining supports cleaner few-shot comparisons in retrieval-augmented systems.
How it connects
Connects T5-v1.1 to sequence-to-sequence models, retrieval augmentation, dual encoders, and parametric versus non-parametric knowledge.

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.