liveBook · Manning

Decoding strategy

As explained in Applied Reinforcement Learning

Decoding is the procedure that converts a model’s output scores into a selected token or sequence.

At each generation step, a language model assigns a score, or logit, to every possible next token. Decoding converts those scores into a probability distribution and selects a token. The selected token is appended to the sequence, and the process repeats. section 10.2

The next token can be selected in different ways:

  • Choose the highest-probability token. section 10.2
  • Sample from the probability distribution to allow more varied responses. section 10.2

In reinforcement learning, the language model can be treated as a stochastic policy over tokens: it generates responses that a reward model scores, and the policy is updated to make desirable responses more probable. section 10.2

How Applied Reinforcement Learning covers it

How deep it goes
Traces logits through probability conversion and token choice, then repeats after appending each token; leaves detailed architecture unexpanded.
Examples
Contrasts choosing the highest-probability token with sampling from the distribution.
What you can do after
Understand how next-token scores become iterative text generation.
How it connects
Links decoding to stochastic policies, reinforcement learning, reward models, and Markov decision processes.

Also explained in

Compare how 14 books explain decoding strategy →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.