A chat completion API generates an assistant response from an ordered list of messages. It suits applications such as chatbots and writing assistants, where a user may ask follow-up questions. Unlike a single prompt sent to a completion API, the message list can include earlier turns so the model can respond in context. section 3.0 section 3.4
Each message has a role and text content:
- system sets high-level behavior, instructions, and response format. section 3.4
- user supplies a question or task. section 3.4
- assistant supplies a prior reply when continuing a conversation. section 3.4
A single-turn task through chat completion
With an AzureOpenAI client already configured, a nonchat task can use the same interface: response = client.chat.completions.create( model="gpt-35-turbo", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Translate 'Hello, how are you?' to Spanish."} ] ) For a follow-up, include the relevant earlier user and assistant messages in order. This abbreviated call shows the request structure, not a complete production application. section 3.4.3 section 3.4
Table 3.6 Chat completion API role description
|
Role parameter
|
Description
|
|---|---|
system
|
The
system role is typically used to set the assistant’s behavior and provide the model with high-level instructions that guide the behavior throughout the conversation. This is where we can describe the assistant’s personality and tell it what it should and should not answer, as well as how to format responses. While there is no token limit, it is included with every API call and is part of the overall token limit.
|
user
|
This represents the user’s input in the conversation; these messages contain the instructions or queries from the user that the assistant responds to.
|
assistant
|
This represents the assistant’s prior messages in the conversation. Think of this as the ongoing memory that helps the model and provides the conversation context as it proceeds, turn by turn.
|
Use the role table when deciding which instructions and prior turns belong in the message list. A system message also counts toward the request’s token limit. section 3.4
Conversation history consumes tokens, as does the generated response. Track the combined budget before each call; when a conversation grows too long, omit, shorten, or truncate messages to fit the model’s limit. Refine instructions and generation parameters through testing rather than assuming the first response is suitable. Inspect the response’s finish reason when diagnosing why generation ended. section 3.4.4 section 3.5
The same API can generate text for nonchat tasks and form the generation step of retrieval-augmented Q&A. In the illustrated blog-data application, embeddings and Redis search retrieve matching posts before the chat call produces an answer. That integration alone is not production-ready: deployment, scaling, security, and operations still need engineering. section 3.4.3 section 8.6
