liveBook · Manning

Counter

As explained in Designing AI Systems

A counter metric is a monotonically accumulating measurement of events or quantities over time.

A counter is a monotonically accumulating metric for totals such as requests, tokens, or cost. Unlike logs, it provides a pre-aggregated time series for dashboards and operational queries. section 7.4.2

Use counters when each event contributes to an accumulated quantity:

  • Increment request counters to measure throughput over a selected time window. section 7.5.2
  • Accumulate cost counters for cost attribution and budget reporting. section 7.5.2
  • Attach dimensions such as provider, model, workflow ID, and cache status to support filtered queries. section 7.4.2
  • Keep latency and other distribution-sensitive measurements in histograms rather than counters or averages. section 7.4.2

Publishing a request counter

A model-service publisher translates request data into platform metrics: observability.record_counter(PlatformMetrics.MODEL_REQUEST_TOTAL, 1, labels). Repeating this for each handled request produces a running total queryable by the supplied labels. section 7.5.2

Listing 7.4 Platform metric definitions and naming conventions
class PlatformMetrics:
    """Standard metrics emitted by platform services.
    Names follow: ai.platform.{service}.{metric_name}"""
 
    # Model Service
    MODEL_REQUEST_TOTAL = \
        "ai.platform.models.requests_total"
    MODEL_REQUEST_DURATION = \
        "ai.platform.models.request_duration_ms"
    MODEL_TOKENS_PROMPT = \
        "ai.platform.models.tokens.prompt"
    MODEL_TOKENS_COMPLETION = \
        "ai.platform.models.tokens.completion"
    MODEL_COST_USD = \
        "ai.platform.models.cost_usd"
    MODEL_CACHE_HITS = \
        "ai.platform.models.cache_hits_total"    #A
    MODEL_FALLBACKS = \
        "ai.platform.models.fallbacks_total"
 
    # Data Service
    DATA_SEARCH_DURATION = \
        "ai.platform.data.search_duration_ms"
    DATA_RELEVANCE_SCORE = \
        "ai.platform.data.relevance_score"    #B
 
    # Session Service
    SESSION_CONTEXT_TOKENS = \
        "ai.platform.sessions.context_tokens"
    SESSION_TRUNCATIONS = \
        "ai.platform.sessions.truncations_total"
 
    # Guardrails Service
    GUARDRAIL_VIOLATIONS = \
        "ai.platform.guardrails.violations_total"    #C
    GUARDRAIL_LATENCY = \
        "ai.platform.guardrails.evaluation_duration_ms"
 
    # Tool Service
    TOOL_EXECUTIONS = \
        "ai.platform.tools.executions_total"
    TOOL_CIRCUIT_BREAKS = \
        "ai.platform.tools.circuit_breaks_total"

Browse the standardized metric constants and naming convention, including metrics whose _total suffix identifies counters. section 7.4.2

How Designing AI Systems covers it

How deep it goes
Defines accumulation, naming, labels, and publishing, while contrasting counters with histograms rather than detailing storage internals.
Examples
Demonstrates request totals, model cost, cache hits, fallbacks, and a Python publisher calling record_counter with labels.
What you can do after
Apply it via an ObservabilityClient to publish cumulative service measurements with standardized names and query dimensions.
How it connects
Connects counters with metrics, logs, histograms, dimensional labels, observability, dashboards, throughput, and cost attribution.

Also explained in

Compare how 5 books explain counter →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.