Nondeterminism means that repeated executions with the same apparent inputs can produce different outputs. Generative models exhibit this behavior because randomness influences generation, especially at higher temperature settings. section 6.7 section 6.5
Manage nondeterministic generation according to the application:
- Use lower temperature for predictable work such as invoice generation, proposals, and code generation. section 3.2.4
- Use higher temperature when diverse or creative outputs matter, such as marketing content, stories, poems, lyrics, and jokes. section 3.2.4
- Evaluate outputs critically, apply content filtering where appropriate, and check generated images for errors, artifacts, and sensitive implications. section 6.7 section 6.5
Varying completion temperature
A completion request can set temperature=0.8 and n=3 to obtain several creative responses in one call. Lowering temperature to 0.2 produces more similar responses, while 1.8 can produce increasingly incoherent or nonsensical text. section 3.2.4
response = client.completions.create(
model="gpt-35-turbo",
prompt=prompt_startphrase,
temperature=0.8,
max_tokens=100,
n=3,
stop=None)
Inspect the completion call that requests three responses while exposing the temperature setting controlling output variation. section 3.2.4
