When an LLM needs to produce the same response for the same input, greedy decoding offers a simple rule: choose the most probable next token at each step. The resulting output is repeatable rather than randomized. section 7.2.1 section 12.3
This is useful when generating candidate phrases for chatbot intent data or summaries that should stay consistent. Sampling decoding instead produces randomized responses, which can help when exploring a variety of outputs. section 7.4 section 12.3
Generating chatbot phrases
For a chatbot helping someone who forgot a password, an LLM can generate phrases such as “I lost my password” and “I cannot remember my password.” Greedy decoding makes the output repeatable for the same prompt. section 7.2.1
