Greedy decoding generates autoregressive text by selecting the highest-probability next token for the current context, then repeating that choice until reaching a length limit or termination token. section 5.2.1 section 5.7
The selection rule chooses the token w with maximum conditional probability given all preceding tokens. section 5.2.1
Apply the procedure one token at a time:
- Initialize the sequence with a prompt or first token. section 5.2.1
- Predict the probability distribution for the next token from the current context. section 5.2.1
- Select the highest-probability token and append it to the sequence. section 5.2.1
- Repeat prediction and selection until the desired length or an end-of-sequence token is reached. section 5.2.1
Choosing the next word
Given “The cat sat on the...” and candidate probabilities for “jumps,” “runs,” and “walks,” greedy decoding appends whichever candidate has the highest probability, then predicts again from the expanded sentence. section 5.2.1
The method is computationally efficient and simple to implement because it follows only one path. That determinism can produce repetitive, monotonous text and prevents exploration of alternative continuations. section 5.2.1
Use greedy decoding when efficient, deterministic generation matters; compare it with beam search, top-k sampling, nucleus sampling, and temperature scaling when diversity or creativity matters. section 5.2 section 5.5















