liveBook · Manning

Reflection token

As explained in Retrieval Augmented Generation, The Foundational Ideas

IsUse is a Self-RAG reflection token that signals the overall usefulness of a generated answer.

A retrieved answer can be well supported and still fail to answer the question. Self-RAG uses reflection tokens—special vocabulary items emitted alongside generated text—to make judgments about retrieval, evidence, and answer quality machine-readable. IsUse is the final quality gate: it rates the usefulness of the full answer to the query from 1 (useless) to 5 (perfect). section 9.2 section 9.2.2

Table 9.1 The four Self-RAG reflection tokens
Token
Input
Output values
What the model is asserting
Retrieve
query (and partial generation)
Yes, No, Continue
Whether to call the retriever, skip it, or keep using evidence already in context
IsRel
query, retrieved passage
Relevant, Irrelevant
Whether the passage carries useful information for answering the query
IsSup
query, passage, generated sentence
Fully supported, Partially supported, No support
Whether every verification-worthy claim in the sentence is backed by the passage
IsUse
query, full generation
1, 2, 3, 4, 5
How useful the final response is as an answer to the query

Compare IsUse’s whole-answer judgment with the other three Self-RAG decisions: whether to retrieve, whether a passage is relevant, and whether a generated sentence is supported. section 9.2.2

Supported but unhelpful

Suppose an answer accurately summarizes a retrieved source but focuses on the wrong aspect of it. Its sentences may pass the IsSup support check while the full response receives a low IsUse score. Grounding and usefulness are separate tests. section 9.2.2

In trained Self-RAG, a model learns to predict reflection tokens from critic-annotated examples. At inference time, token-guided decoding can favor candidate continuations with higher predicted usefulness alongside relevance and support. Adjusting the token weights changes that preference without retraining; it does not make the model’s self-judgment infallible. section 9.3 section 9.3.2

IsUse also serves as a monitoring signal. Log its production distribution and compare sampled scores with human or stronger-model ratings using rank correlation. Low IsUse scores when relevance and support look healthy can point to missing coverage of the user’s question. Because a model grading itself can share the assumptions behind its answer, retain external evaluation. section 9.2.3 section 9.8

A prompted workflow can emulate the critique with separate model calls instead of training new tokens. That is easier to set up but adds an API round-trip per critique; trained Self-RAG moves the judgment into the model at a greater training and deployment cost. section 9.4

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Defines the five-point scale and explains its roles in decoding, monitoring, and calibration without specifying a separate IsUse scoring rubric.
Examples
Contrasts fully supported sentences with an unhelpful answer that summarizes the wrong aspect of a source.
What it assumes
Assumes familiarity with language-model tokens and retrieved passages while defining reflection tokens and their distinct decisions.
What you can do after
Diagnose/debug with it by tracking IsUse distributions and checking sampled scores against external ratings.
How it connects
Connects answer usefulness to Self-RAG, retrieval relevance, groundedness, token-guided decoding, and offline evaluation.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.