AI-specific metrics extend infrastructure monitoring with measurements that reveal whether an AI system is useful, safe, appropriately routed, and economically sustainable. Uptime, latency, and errors can remain normal while response quality or retrieval relevance deteriorates. section 7.1.1
Production observability captures dimensions such as:
- Token usage and cost attribution, exposing consumption and spending patterns. section 7.1.1
- Model-selection visibility, revealing unexpected routing to cheaper or less capable models. section 7.1.1
- Retrieval-quality metrics, indicating whether returned document chunks provide relevant context. section 7.1.1
- Safety-filter activations and response-quality scoring, distinguishing real problems from false positives and weak answers. section 7.1.1
Figure 7.3 AI observability extends traditional monitoring with dimensions unique to AI systems. Traditional metrics (uptime, latency, errors) still matter but miss the questions that AI operations teams need answered. Token tracking, model selection visibility, retrieval quality, safety evaluations, cost attribution, and response quality scoring are all AI-specific concerns that the platform must capture.

Maps traditional monitoring metrics to AI-specific dimensions, including token tracking, model selection, retrieval quality, safety evaluation, cost attribution, and response quality. section 7.1.1
