A system prompt is the initial instruction supplied in a chat completion’s system message. It sets the assistant’s behavior before user messages arrive. It can describe the assistant, specify its personality, establish rules, and provide information needed for a task. Although the API makes the system role optional, setting it deliberately is recommended. section 3.4.1
A useful system prompt makes the intended behavior explicit:
- State the assistant’s job and scope, such as answering questions about pets rather than unrelated topics. section 6.4.1
- Specify the response format and show an example when structured output matters; refine the instruction if the model adds unwanted fields. section 6.4.1
- Provide small amounts of relevant context and say what to do when the answer is unknown. For larger bodies of information, retrieve relevant passages instead of placing everything in the system message. section 3.4.1
Ground a technical-support assistant
Place an instruction such as “Only answer using the context below. If you are unsure, say ‘I don’t know’” in a system message, followed by a short service FAQ. Send the customer’s question as a user message. The system message supplies the answering rule and reference information; the user message supplies the immediate task. section 3.4.1
Listing 6.2 Prompt engineering example
import os
import openai
client = AzureOpenAI(
azure_endpoint=os.getenv("AOAI_ENDPOINT"),
api_version="2024-05-01-preview",
api_key=os.getenv("AOAI_KEY")
)
GPT_MODEL = "gpt-35-turbo"
conversation=[{"role": "system", "content": "You are an AI
assistant that extracts entities from text
as JSON. \nHere is an example of your output
format:\n{ \n \"the_name\": \"\",\n
\"the_company\": \"\",\n \"a_phone_number\":
\"\"\n}"}]
print("Please enter what you want to talk about:")
while True:
user_input = input()
conversation.append({"role": "user", "content": user_input})
response = openai.ChatCompletion.create(
model = GPT_MODEL,
messages = conversation
)
conversation.append({"role": "assistant", "content":
response["choices"][0]["message"]["content"]})
print("\nAI:" + response['choices'][0]['message']['content'] + "\n")
The API listing shows a system message containing an entity-extraction instruction and JSON output example, followed by user input in a continuing conversation. section 6.4.1
Treat the prompt as an instruction to test, not a guarantee. Iterate when outputs miss the requested format or behavior; even a fine-tuned model can need a system-prompt adjustment. In a continuing conversation, count both messages and expected output against the model’s token limit, shortening context when necessary. Review sensitive-content handling as part of the application design. section 9.4.1 section 3.4.4
