Temperature scaling adjusts the next-token probability distribution before sampling by dividing log-probabilities by a temperature value. It controls the balance between deterministic, repetitive text and more creative, random output. section 5.2.3
Use the temperature value to change sampling behavior:
- Set T > 1 to increase randomness and encourage more surprising, creative outputs, while accepting a greater risk of irrelevant or nonsensical text. section 5.2.3
- Set T < 1 to make generation more deterministic by favoring the most probable tokens. section 5.2.3
- Move T close to 0 to approach greedy decoding. section 5.2.3
- Treat the temperature as a smooth randomness control, but tune it carefully because high values can produce gibberish and low values can limit creativity. section 5.2.3
Low versus high temperature
For a prompt such as "The quick brown fox," a low temperature favors the most probable continuation and produces more repeatable text. A high temperature explores less-probable continuations, increasing diversity while potentially reducing coherence. section 5.2.3
The probability transformation divides each token's log-probability by T, exponentiates the result, and normalizes across candidate tokens before sampling. section 5.2.3

