Azure OpenAI is a managed service for using OpenAI models in applications. Microsoft handles the underlying infrastructure and maintenance; the service runs on Azure for scalability, reliability, and regional availability. Compared with direct OpenAI access, it is aimed at enterprises that need stronger security, privacy, and regulatory controls. The underlying models are the same, but the APIs are similar rather than identical. section 2.7.1 section 2.7.2
For a chat application that answers questions from organizational data, Azure OpenAI on your data provides a managed alternative to assembling and operating the entire retrieval stack yourself. It supports running chat models over connected data, with scalability, security, refreshes, and integration with other Azure services. This can shorten the path to deployment, though data-source and ingestion choices still need to be configured. section 8.6 section 8.7
A practical setup starts with these choices:
- Obtain an Azure subscription and endpoint access. Keep the API key and endpoint out of source code; environment variables are one demonstrated option. section 3.1.1
- Inspect available models and their capabilities before selecting one for chat, text completion, embeddings, or fine-tuning. The available list can differ by organization. section 3.1.2
- For chat over your data, connect a source through Azure AI Studio or the REST API. Supported Studio options include an Azure AI Search index, Blob storage, Cosmos DB, and a URL. Configure the storage resource, search resource, index, and embedding details as needed. section 8.7
Listing 3.1 Listing Azure OpenAI models available
import os
import json
from openai import AzureOpenAI #1
client = AzureOpenAI(
azure_endpoint=os.getenv("AOAI_ENDPOINT"), #2
api_version="2023-05-15", #3
api_key=os.getenv("AOAI_KEY") #4
)
# Call the models API to retrieve a list of available models
models = client.models.list()
# save to file
with open('azure-oai-models.json', 'w') as file:
models_dict = [model.__dict__ for model in models]
json.dump(models_dict, file)
# Print out the names of all the available models, and their capabilities
for model in models:
print("ID:", model.id)
print("Current status:", model.lifecycle_status)
print("Model capabilities:", model.capabilities)
print("-------------------")
The Python listing connects to an Azure endpoint using environment variables, calls the models API, and inspects available models and capabilities. Its API version is specific to the example, not a universal setting. section 3.1.2
Production requirements remain part of the decision. Azure OpenAI offers data privacy, customer-managed keys, encryption at rest, private networking, regional availability, and responsible-AI content filtering. Using an existing Azure AI Search index can also shift the work of keeping ingestion current from a self-managed search setup to an Azure service. Choose the service for those operational needs, not because connecting a model alone removes the need to plan data integration. section 2.7.2 section 8.7
