Resource utilization measures the CPU, memory, and capacity consumed by a running application. In production API deployments, these resources have configured limits, so monitoring them helps preserve performance and allocate capacity effectively. section 11.1.1
Basic resource monitoring connects deployment constraints to operational decisions:
- Track CPU and memory against the limits assigned during deployment. section 11.1.1
- Consider pod-scaling limits when evaluating whether an application can absorb additional load. section 11.1.1
- Scrape BentoML's built-in /metrics endpoint with Prometheus and visualize utilization in Grafana. section 11.1.1
- Review utilization alongside response latency and failing request counts to identify performance and reliability problems. section 11.1.1
Figure 11.5 BentoML basic monitoring dashboard
Use the BentoML basic monitoring dashboard to inspect resource and request metrics together. section 11.1.1
