A horizontally scaled RAG service cannot keep chat history or feedback on a container’s local disk: restarts erase it, and replicas cannot share it. Cosmos DB supplies external persistence for small JSON records, keeping the service stateless while retaining a durable, queryable trail for audits and debugging. section 5.1.1 section 5.1.3
Figure 5.2 The RAG container keeps no secrets and writes nothing to its own disk; it simply streams prompts from the browser, fans out to AI Search and OpenAI for retrieval and generation, and drops a JSON record into Cosmos DB for posterity. All configuration arrives at launch via environment variables or a key vault. The dashed line to “Local Disk” drives home the point: local storage is strictly off-limits in a truly stateless service.
The deployment flow makes the boundary clear: the container streams and serves requests, while Cosmos DB receives persistent JSON records and local disk remains unused. section 5.1.1
For chat logging, create a Cosmos client from a URI and key held in environment variables, select a database and container, then upsert one document per response. A record can include a generated id, user question, timestamp, final answer, email, and agent search results. The SDK handles are lightweight enough to retain as reusable application-level clients. section 5.1.3 section 5.5
Persisting a chat response
A response stream can continue to the user while its associated question, answer, timestamp, and retrieval results are packaged as JSON and sent over HTTPS to a ragchat_logs container. Later queries support debugging slow responses, auditing behavior, and refining ranking. section 5.1.3
Cosmos DB also fits a feedback pipeline because schemaless JSON permits new fields without column migrations. Store thumbs-up or thumbs-down ratings, comments, the original question and answer, and sender email in a user_feedback container; a partition key of /id follows the same per-record pattern. section 6.3 section 6.7
Figure 6.4 The relationship between your web browser and CosmosDB. We have a function app in the middle for database security.
A separate Azure Function sits between browser and database, keeping Cosmos DB credentials out of client-side code while isolating quick feedback writes from long-lived chat streaming. section 6.5
Use a server-side feedback write path:
- Receive the feedback payload in an HTTP-triggered Azure Function and extract the question, answer, rating, reason, and email. section 6.7
- Load the Cosmos endpoint, key, database name, and container name from environment configuration rather than embedding secrets in browser code. section 6.5 section 6.7
- Write a new JSON item to the feedback container, then test the local endpoint with a REST client before deployment. section 6.7
Do not connect a web page directly to Cosmos DB. A client-side key can be exposed through browser developer tools, allowing unwanted database access, deletion, or spending. Keep credentials server-side with environment variables, Key Vault, and role-based access control; split feedback writes into a Function so security and scaling remain independent of the chat workload. section 6.5
Operationally, query negative feedback by rating and time to inspect recent complaints, track the thumbs-up-to-thumbs-down ratio, and review the feedback store regularly. Alerts through Azure Monitor can flag spikes in negative ratings. section 6.11

