Temperature controls how much randomness a language model uses when choosing output tokens. A lower value makes high-scoring tokens more likely; a higher value allows more varied choices. OpenAI accepts values from 0 to 2, with a default of 1. section 3.5.3
Choose a value to suit the task:
- For a story, try a higher temperature when you want more variation. A story-generation example uses 1.5. section 3.5.4
- For review classification, use a low temperature when repeatable output matters. A classifier uses 0 to minimize randomness while requesting only “pos” or “neg.” section 9.2
- Experiment rather than assuming more or less randomness is always better: too much can produce output that ignores the input or instructions, while too little can limit diversity. section 3.5.3
Listing 3.3 Using GPT-4o for chat completion with custom parameter settings
import openai
client = openai.OpenAI()
result = client.chat.completions.create(
model='GPT-4o',
messages=[{
'role':'user',
'content':'Tell me a story!'
}],
max_tokens=512, #1
stop='happily ever after', #2
temperature=1.5, #3
presence_penalty=0.5, #4
logit_bias={14844:-100}) #5
print(result.choices[0].message.content)
This Python chat-completion call sets temperature to 1.5 for a story. It also limits the response length, supplies a stop phrase, and configures other output controls. section 3.5.4
Temperature is not the only randomization control. The alternative top_p setting limits which tokens are eligible for selection; avoid changing both settings in the same model call. section 3.5.3
