RAG-Sequence is a retrieval-augmented generation pattern that commits the entire answer to one retrieved document. It evaluates complete answers conditioned separately on the top-K documents, then uses their sequence probabilities to choose or marginalize over candidates. section 4.3 section 4.3.1
Its central trade-offs are:
- It is efficient for questions whose answer resides in one coherent passage, such as asking for AWS revenue. section 4.3.1
- It can miss multi-document answers because one generated sequence cannot combine evidence from separate documents. section 4.3.1
- Compared with RAG-Token, it places document marginalization outside the product over generated tokens and requires document-specific decoding. section 4.3.3
- Its accuracy is competitive on several open-domain question-answering datasets, while token-level mixing is more useful when answers genuinely draw on multiple documents. section 4.3.3
A comparative earnings question
For a question comparing AWS and Azure revenue, RAG-Sequence may generate one answer from Amazon's report and another from Microsoft's, then favor the higher-probability answer. The result can summarize one company accurately while omitting the requested comparison. section 4.3.1
Table 4.1 RAG sequence and RAG-token trade-offs
|
Model variant
|
NQ
|
TriviaQA
|
WebQuestions
|
CuratedTrec
|
|
RAG-Token
|
44.1
|
55.2
|
45.5
|
50.0
|
|
RAG-Sequence
|
44.5
|
56.8
|
45.2
|
52.2
|
Use the trade-off table to compare RAG-Sequence and RAG-Token across four open-domain question-answering datasets. section 4.3.3
