Tune a language model’s generation settings to constrain its output, improve classification results, and avoid generating tokens that will not be used. section 9.3
Classifying reviews
For a ten-review classification sample, limiting output to one token increased correct labels from six to seven. This works because each of the two possible labels fits in one token. section 9.3
response = client.chat.completions.create(
model='gpt-3.5-turbo',
messages=[
{'role':'user', 'content':prompt}
],
logit_bias = {981:100, 29875:100}, #1
temperature=0, max_tokens=1
)
See the OpenAI Python call that combines token restrictions with a one-token output limit. section 9.3
