A retrieved answer can be well supported and still fail to answer the question. Self-RAG uses reflection tokens—special vocabulary items emitted alongside generated text—to make judgments about retrieval, evidence, and answer quality machine-readable. IsUse is the final quality gate: it rates the usefulness of the full answer to the query from 1 (useless) to 5 (perfect). section 9.2 section 9.2.2
Table 9.1 The four Self-RAG reflection tokens
|
Token
|
Input
|
Output values
|
What the model is asserting
|
|
Retrieve
|
query (and partial generation)
|
Yes, No, Continue
|
Whether to call the retriever, skip it, or keep using
evidence already in context
|
|
IsRel
|
query, retrieved passage
|
Relevant, Irrelevant
|
Whether the passage carries useful information for
answering the query
|
|
IsSup
|
query, passage, generated sentence
|
Fully supported, Partially supported, No support
|
Whether every verification-worthy claim in the sentence is
backed by the passage
|
|
IsUse
|
query, full generation
|
1, 2, 3, 4, 5
|
How useful the final response is as an answer to the query
|
Compare IsUse’s whole-answer judgment with the other three Self-RAG decisions: whether to retrieve, whether a passage is relevant, and whether a generated sentence is supported. section 9.2.2
Supported but unhelpful
Suppose an answer accurately summarizes a retrieved source but focuses on the wrong aspect of it. Its sentences may pass the IsSup support check while the full response receives a low IsUse score. Grounding and usefulness are separate tests. section 9.2.2
In trained Self-RAG, a model learns to predict reflection tokens from critic-annotated examples. At inference time, token-guided decoding can favor candidate continuations with higher predicted usefulness alongside relevance and support. Adjusting the token weights changes that preference without retraining; it does not make the model’s self-judgment infallible. section 9.3 section 9.3.2
IsUse also serves as a monitoring signal. Log its production distribution and compare sampled scores with human or stronger-model ratings using rank correlation. Low IsUse scores when relevance and support look healthy can point to missing coverage of the user’s question. Because a model grading itself can share the assumptions behind its answer, retain external evaluation. section 9.2.3 section 9.8
A prompted workflow can emulate the critique with separate model calls instead of training new tokens. That is easier to set up but adds an API round-trip per critique; trained Self-RAG moves the judgment into the model at a greater training and deployment cost. section 9.4
