liveBook · Manning

Cosmos DB

As explained in Enterprise RAG

Azure Cosmos DB is Microsoft's globally distributed, multimodel cloud database service, including a NoSQL document-database API.

A horizontally scaled RAG service cannot keep chat history or feedback on a container’s local disk: restarts erase it, and replicas cannot share it. Cosmos DB supplies external persistence for small JSON records, keeping the service stateless while retaining a durable, queryable trail for audits and debugging. section 5.1.1 section 5.1.3

Figure 5.2 The RAG container keeps no secrets and writes nothing to its own disk; it simply streams prompts from the browser, fans out to AI Search and OpenAI for retrieval and generation, and drops a JSON record into Cosmos DB for posterity. All configuration arrives at launch via environment variables or a key vault. The dashed line to “Local Disk” drives home the point: local storage is strictly off-limits in a truly stateless service.

The deployment flow makes the boundary clear: the container streams and serves requests, while Cosmos DB receives persistent JSON records and local disk remains unused. section 5.1.1

For chat logging, create a Cosmos client from a URI and key held in environment variables, select a database and container, then upsert one document per response. A record can include a generated id, user question, timestamp, final answer, email, and agent search results. The SDK handles are lightweight enough to retain as reusable application-level clients. section 5.1.3 section 5.5

Persisting a chat response

A response stream can continue to the user while its associated question, answer, timestamp, and retrieval results are packaged as JSON and sent over HTTPS to a ragchat_logs container. Later queries support debugging slow responses, auditing behavior, and refining ranking. section 5.1.3

Cosmos DB also fits a feedback pipeline because schemaless JSON permits new fields without column migrations. Store thumbs-up or thumbs-down ratings, comments, the original question and answer, and sender email in a user_feedback container; a partition key of /id follows the same per-record pattern. section 6.3 section 6.7

Figure 6.4 The relationship between your web browser and CosmosDB. We have a function app in the middle for database security.

A separate Azure Function sits between browser and database, keeping Cosmos DB credentials out of client-side code while isolating quick feedback writes from long-lived chat streaming. section 6.5

Use a server-side feedback write path:

  • Receive the feedback payload in an HTTP-triggered Azure Function and extract the question, answer, rating, reason, and email. section 6.7
  • Load the Cosmos endpoint, key, database name, and container name from environment configuration rather than embedding secrets in browser code. section 6.5 section 6.7
  • Write a new JSON item to the feedback container, then test the local endpoint with a REST client before deployment. section 6.7

Do not connect a web page directly to Cosmos DB. A client-side key can be exposed through browser developer tools, allowing unwanted database access, deletion, or spending. Keep credentials server-side with environment variables, Key Vault, and role-based access control; split feedback writes into a Function so security and scaling remain independent of the chat workload. section 6.5

Operationally, query negative feedback by rating and time to inspect recent complaints, track the thumbs-up-to-thumbs-down ratio, and review the feedback store regularly. Alerts through Azure Monitor can flag spikes in negative ratings. section 6.11

How Enterprise RAG covers it

How deep it goes
Implements Python SDK clients, JSON upserts, container setup, Azure Function writes, queries, and credential handling without database internals.
Examples
Uses chat-response logs, thumbs-down product feedback, a user_feedback container, Python upserts, and a query for recent negative ratings.
What it assumes
Assumes a deployed RAG service and Azure account while walking through Cosmos containers, credentials, and local Function testing.
What you can do after
Operate/configure it to persist chat and feedback JSON through server-side Python and Azure Functions.
How it connects
Connects stateless containers, RAG, Azure Functions, environment variables, Key Vault, role-based access control, and Azure Monitor.

Also explained in

Compare how 2 books explain Cosmos DB →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Enterprise RAG. The section links point to the parts of the book it draws on.