Azure App Service provides a managed public host for a containerized web application. It runs the image without requiring application owners to manage virtual machines, operating-system patches, or SSL certificates, while exposing an HTTPS URL for users. section 5.2.2
For a streaming RAG chatbot, App Service is the production step after a Docker image works locally. The service can keep WebSocket connections open, accept public traffic, and add replicas when demand rises. That makes it a better fit here than relying on a developer machine, a brittle VM, or a cold-start-prone serverless deployment. section 5.0 section 5.2.2
A production deployment follows a deliberate path:
- Package the application as a self-contained Docker image so it boots consistently across local and hosted environments. section 5.0
- Store the image in a container registry, giving App Service an immutable artifact to pull. section 5.2
- Create a Web App configured for Container publishing, Linux, an appropriate region, public access, and an App Service plan suited to the workload. section 5.2.2
- Put secrets in the Web App configuration rather than a local .env file or the image, and connect deployment to GitHub Actions so commits follow the same promotion path. section 5.2
- Enable Application Insights and add application logging before load testing, so slow or failed requests can be traced to a bottleneck. section 5.2.2 section 5.5
Scaling a slow chatbot
When requests become slow or stop returning, first inspect Web App logs and identify the constrained component. If the application instance lacks capacity, increase its CPU or memory, or use Scale out in the App Service plan to set a higher instance count. Azure then distributes incoming traffic across the instances; plan limits may restrict the count. section 5.4.3 section 5.5
Scaling the web tier is only one response to load-test failures. A chatbot can also be constrained by AI Search throughput or OpenAI quota, so telemetry should guide whether to add App Service instances or scale the dependency that is actually throttling requests. This keeps response times responsive without scaling every component unnecessarily. section 5.4.3 section 5.5
Figure 5.1 This flowchart shows the key stages of this chapter. We will start by packaging our app as a container, then deploy it to Azure’s app service, then test it remotely. Finally, we will perform load testing using logging and simulated users with Locust.
Follow the deployment loop from container packaging through App Service deployment, remote testing, logging, and simulated-user load testing. section 5.0
