liveBook · Manning

Prometheus Adapter

As explained in LLMs in Production

Prometheus Adapter is a Kubernetes metrics adapter that exposes Prometheus metrics through Kubernetes metrics APIs.

Suppose a deployment needs to scale on GPU utilization rather than CPU or memory. A Kubernetes Horizontal Pod Autoscaler (HPA) can already access CPU and memory through its built-in metrics endpoint, but it needs a registered custom or external metrics API service to access other metrics. Prometheus Adapter can expose Prometheus metrics through that API for autoscaling. section 6.2.2

The trade-off is configuration and flexibility. Prometheus Adapter requires metrics to be configured in a ConfigMap. KEDA can use metrics already exposed through the Prometheus API and can scale a service to or from zero on suitable signals, such as traffic or queue metrics. Scaling from zero does not work with CPU, memory, or GPU resource metrics. Only one adapter API service can be registered at a time for each metrics API, which complicates switching providers. section 6.2.2

How LLMs in Production covers it

How deep it goes
Explains the adapter’s role and ConfigMap requirement, but does not provide installation steps or configuration.
Examples
Uses GPU utilization as a custom autoscaling signal and contrasts traffic or queue metrics for scaling from zero.
What you can do after
Evaluate/understand when Prometheus Adapter fits custom-metric autoscaling and when KEDA offers a more flexible alternative.
How it connects
Links Prometheus, Horizontal Pod Autoscalers, custom metrics APIs, ConfigMaps, and KEDA.
Cover of LLMs in Production
liveBookmanning.com
Read LLMs in Production on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.