A token limit sets the maximum amount of tokenized text a model may generate or process in one interaction. Because tokens are words or word pieces, the setting does not directly count requested items such as keywords. A prompt and its expected response must fit within the available limit; otherwise, the response may be incomplete or cut off. section 5.5.3 section 1.1
Twenty keywords need more than twenty tokens
Suppose an API request asks for 20 Portuguese keywords expressing positive sentiment in customer reviews. Setting max_tokens=20 may produce a truncated list because the model needs tokens for each keyword and the surrounding response. Testing values below 100, or even 120, can still produce incomplete output; the suitable value depends on the response format and content. section 5.5.3
Use token limits as a response-management setting, not as a direct item counter:
- Leave room for the complete answer rather than matching the requested number of words or items one-for-one. section 5.5.3
- Inspect the returned format manually or strengthen the prompt when consistent output matters. section 5.5.3
- Split long inputs or requests into smaller portions when the response is truncated or processing becomes difficult. section 5.3.2
- Check current API documentation because parameter names and interfaces may change. section 5.5.3
Listing 5.5 Generating keywords via ChatGPT API
from openai import OpenAI
def generate_keywords(temperature=0.5, max_tokens=150):
client = OpenAI(
api_key="your_api_key", # Make sure to use your actual API key here
)
prompt = """Generate a list of 20 keywords indicating positive
sentiment to be used for searching customer reviews in Portuguese."""
try:
response = client.chat.completions.create(
messages=[
{
"role": "user",
"content": prompt,
}
],
model="gpt-4-0125-preview", # Replace with the model you have
access to
temperature=temperature,
max_tokens=max_tokens,
)
return(response.choices[0].message.content.split("\n"))
except Exception as e:
print(f"An error occurred: {str(e)}")
# Example usage
print("Generated Keywords:")
print(generate_keywords())
Shows max_tokens alongside temperature in a keyword-generation workflow, including response extraction and error handling. section 5.5.3
