Beam search is a heuristic decoder that avoids greedy decoding's single-path commitment. At each step, it keeps the top k partial sequences, extends each by one token, prunes back to k candidates, and returns the highest-scoring completed sequence. section 9.3.2
In Self-RAG, beam search scores candidates by combining ordinary text probability with weighted probabilities for positive reflection tokens. Relevance, support, and usefulness can boost a continuation; self-identified lack of support can penalize and remove it. Runtime weights trade fluency against factual grounding without retraining. section 9.3.2
Beam width and retrieval design create practical trade-offs:
- RAG-token uses one standard beam search while allowing different generated tokens to rely on different retrieved documents. section 4.3.3
- RAG-sequence requires separate beam searches for documents and additional forward passes for hypotheses missing from a beam. section 4.3.3
When evidence is split across documents
For a response naming a city and a state, token-level decoding can use one passage for the city and another for the state. Sequence-level decoding commits to one passage for the complete answer, so token-level scoring can be substantially higher when evidence is distributed. section 4.3.3
Use the scoring equation to inspect how ordinary text likelihood and reflection-token likelihood combine into a beam-ranking score. section 9.3.2






