Alerting generates and delivers notifications when monitored applications, systems, or infrastructure show failures, degradation, security events, or other conditions requiring intervention. It turns monitoring data into proactive incident management: responsible teams can act before faults cause extended downtime or user impact. section 11.1 section 11.1.4
A production alerting setup typically combines:
- Prometheus alert rules expressed as PromQL conditions over monitored metrics. section 11.1.4
- Service-availability checks using the up metric, plus absent when the target or metric disappears entirely. section 11.1.4
- Alertmanager routing that groups alerts and sends them through configured channels such as email, Slack, or PagerDuty. section 11.1.4
Detecting a missing deployment
A BentoML deployment can trigger ServiceDown when up equals 0 and MissingUpMetric when absent(up{...}) equals 1. Requiring the condition for five minutes and assigning critical severity reduces transient noise while preserving urgent notification. Alertmanager can group alerts by alertname and job, then route them to Gmail with resolved notifications enabled.
