Temperature is a decoding parameter that reshapes the next-token probability distribution. Sampling remains probabilistic, while temperature controls whether generation favors predictable or surprising continuations. section 5.2.3
Adjusting temperature changes the balance between determinism and diversity:
- T > 1 flattens the distribution, increasing randomness and creative variation while raising the risk of irrelevant or nonsensical text. section 5.2.3
- T < 1 sharpens the distribution, favoring highly probable words and producing more deterministic output. section 5.2.3
- A temperature close to zero approaches greedy decoding, which selects the most probable continuation rather than exploring alternatives. section 5.2.3
Same prompt, different behavior
For the same next-token distribution, a low temperature repeatedly favors the leading continuation, whereas a high temperature gives less probable alternatives more opportunity to be selected. This makes temperature useful for tuning predictable versus creative generation. section 5.2.3
The distribution is computed by dividing each token's log-probability by T and normalizing the exponentiated values. section 5.2.3



















