Temperature adjusts token probabilities before sampling. Higher values make the distribution flatter and outputs more varied; lower values make it sharper and outputs more consistent. It influences generation but does not impose a strict limit. section 2.1.3
Temperature affects which token the model selects. Unlike max_tokens, it does not cap response length; unlike API settings such as authentication and retries, it changes generation rather than the call itself. section 2.1.3
Choose a setting to suit the agent’s role:
- Use a low temperature when consistent answers or code are important. section 2.1.3
- Use a higher temperature for creative writing or brainstorming, accepting less factual accuracy. section 2.1.3
- For routine planning, temperature 0 can make outputs more consistent and easier to evaluate, but does not guarantee identical answers. section 2.3.2
Table 2.1 LLM parameters and how they can be used to control output
| Parameter |
Typical range |
What it really does |
When to raise / lower |
|---|---|---|---|
| Temperature |
0–2 |
Scales every token’s probability before sampling. Higher = flatter distribution ⇒ more randomness/creativity. Lower = sharper ⇒ more deterministic, repetitive output. |
Raise for brainstorming, fiction. Lower for consistent answers, code. |
| top-p (nucleus) |
0–1 |
Keeps only the smallest set of tokens whose cumulative probability ≥ p; renormalizes, then samples. Controls variety without letting low-probability “tail” tokens in. |
Lower(< 0.9) for tight, on-topic prose. Higher (~1) for free-form generation. |
| max_tokens |
1–8192 + (model-dependent) |
A hard ceiling on how many tokens the model may generate in the response. (Prompt tokens + max ≤ context window.) |
Set just above the expected length. It prevents rambling and keeps cost down. |
| presence_penalty |
–2 … +2 |
Adds a penalty once a token has appeared; encourages new topics |
Use 0.6–1.0 to avoid repeating already-mentioned concepts. |
| frequency_penalty |
–2 … +2 |
Penalizes tokens in proportion to their current count; discourages phrase repetition |
Use 0.2–0.8 to reduce echoing or loops in long answers. |
Compare temperature with top-p and max_tokens, including their effects and typical uses. section 2.1.3
