Randomization adds stochastic variation when mapping token scores to output tokens, allowing repeated generation from the same input to produce different replies. section 3.5.3
It balances diversity against usefulness: too little randomization makes outputs repetitive, while too much can disconnect them from the input and instructions. Choosing the right degree may require experimentation. section 3.5.3
Common ways to control randomization include:
- Temperature: higher values increase randomization; lower values make the highest-scoring token more likely. Values range from 0 to 2, with a default of 1. section 3.5.3
- Top_p: an alternative parameter for controlling how token scores become output choices. section 3.5.3
Regenerating an answer
When a ChatGPT reply is unsatisfactory, select Regenerate to request another answer to the same input. Randomization makes the second output likely to differ and potentially improve. section 2.2
Figure 2.3 After ChatGPT generates an answer, the Regenerate button appears below the generated response. Click this button to receive an alternative answer to your last input.
Shows where the Regenerate control appears below a generated ChatGPT response. section 2.2
