At each generation step, a language model assigns a score, or logit, to every possible next token. Decoding converts those scores into a probability distribution and selects a token. The selected token is appended to the sequence, and the process repeats. section 10.2
The next token can be selected in different ways:
- Choose the highest-probability token. section 10.2
- Sample from the probability distribution to allow more varied responses. section 10.2
In reinforcement learning, the language model can be treated as a stochastic policy over tokens: it generates responses that a reward model scores, and the policy is updated to make desirable responses more probable. section 10.2













