Consistency means applying the same rules and conventions across records, sources, formats, systems, or repeated model outputs. It asks whether comparable data receives comparable treatment, so analysis does not change merely because its origin or generation run changed. section 2.2 section 5.1.3
Check consistency at several points in an analytical workflow:
- Compare values and representations across sources, formats, and systems before drawing conclusions. section 2.2
- Treat generative-AI output as variable: lists may change between runs or arrive in an unexpected format. section 5.5.3
- Inspect returned data manually, refine the prompt, and use dedicated libraries when repeatable rules, equitable treatment, speed, or scale matter. section 5.1.3 section 5.5.3
A keyword list that changes shape
A prompt requests 20 Portuguese keywords expressing positive sentiment in customer reviews. One run may produce a usable list, while another may vary its terms or formatting. A Python workflow can split the response into lines, but the result still requires inspection, prompt refinement, and checks for truncation. The max_tokens setting limits response tokens rather than the number of keywords, so a setting below roughly 100 or 120 can produce an incomplete list. section 5.5.3
Listing 5.5 Generating keywords via ChatGPT API
from openai import OpenAI
def generate_keywords(temperature=0.5, max_tokens=150):
client = OpenAI(
api_key="your_api_key", # Make sure to use your actual API key here
)
prompt = """Generate a list of 20 keywords indicating positive
sentiment to be used for searching customer reviews in Portuguese."""
try:
response = client.chat.completions.create(
messages=[
{
"role": "user",
"content": prompt,
}
],
model="gpt-4-0125-preview", # Replace with the model you have
access to
temperature=temperature,
max_tokens=max_tokens,
)
return(response.choices[0].message.content.split("\n"))
except Exception as e:
print(f"An error occurred: {str(e)}")
# Example usage
print("Generated Keywords:")
print(generate_keywords())
Inspect the Python API workflow that generates keyword lists and exposes the points where formatting, model variation, and token limits can disrupt consistency. section 5.5.3
