liveBook · Manning

RAG-Sequence

As explained in Retrieval Augmented Generation, The Foundational Ideas

RAG-Sequence is a RAG variant that conditions the entire generated sequence on a single retrieved document selected from the candidate set.

RAG-Sequence is a retrieval-augmented generation pattern that commits the entire answer to one retrieved document. It evaluates complete answers conditioned separately on the top-K documents, then uses their sequence probabilities to choose or marginalize over candidates. section 4.3 section 4.3.1

Its central trade-offs are:

  • It is efficient for questions whose answer resides in one coherent passage, such as asking for AWS revenue. section 4.3.1
  • It can miss multi-document answers because one generated sequence cannot combine evidence from separate documents. section 4.3.1
  • Compared with RAG-Token, it places document marginalization outside the product over generated tokens and requires document-specific decoding. section 4.3.3
  • Its accuracy is competitive on several open-domain question-answering datasets, while token-level mixing is more useful when answers genuinely draw on multiple documents. section 4.3.3

A comparative earnings question

For a question comparing AWS and Azure revenue, RAG-Sequence may generate one answer from Amazon's report and another from Microsoft's, then favor the higher-probability answer. The result can summarize one company accurately while omitting the requested comparison. section 4.3.1

Table 4.1 RAG sequence and RAG-token trade-offs
Model variant
NQ
TriviaQA
WebQuestions
CuratedTrec
RAG-Token
44.1
55.2
45.5
50.0
RAG-Sequence
44.5
56.8
45.2
52.2

Use the trade-off table to compare RAG-Sequence and RAG-Token across four open-domain question-answering datasets. section 4.3.3

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Defines document-level marginalization and decoding trade-offs, but stops short of detailed implementation beyond probability formulas and comparisons.
Examples
Uses AWS and Azure earnings reports plus city-and-state evidence split across passages.
What it assumes
Assumes familiarity with retrieval and generation while introducing document marginalization through sequence-versus-token comparison.
What you can do after
Evaluate whether single-document conditioning fits a question and diagnose omissions caused by multi-document evidence.
How it connects
Connects marginalization, retrieval, generation, RAG-Token, beam search, and document mixing.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.