Monitoring observes service telemetry to determine whether a platform is operating acceptably. Production systems need more than inference code: they require visibility into system health, model accuracy, response behavior, and operational conditions. section 1.2.1 section 1.0
Traditional monitoring answers whether servers are available and responses are fast. AI systems also require signals about correctness: an assistant can return successful responses quickly while becoming less helpful, repeating questions, or producing confident errors. section 7.1
A production monitoring design separates several observable concerns:
- Logging records what happened and supports querying events. section 7.3
- Metrics measure quantities and frequencies, while health monitoring evaluates platform-wide service condition from telemetry patterns. section 7.3
- Tracing follows request paths; generations preserve individual language-model calls as first-class records. section 7.3
- Scores capture how well the system worked, while cost management exposes the financial view. section 7.3
A misleadingly healthy deployment
After a new patient-intake prompt is deployed, latency and error rates remain normal, yet patients need more turns and repeat information. Monitoring must correlate infrastructure signals with quality signals rather than treating successful HTTP responses as proof of health. section 7.1
Listing 7.1 Observability Service gRPC definition
syntax = "proto3";
package ai.platform.observability.v1;
service ObservabilityService {
rpc IngestLogs(IngestLogsRequest)
➥returns (IngestLogsResponse); #A
rpc QueryLogs(QueryLogsRequest) returns
➥(QueryLogsResponse); #A
rpc RecordMetrics(RecordMetricsRequest)
➥returns (RecordMetricsResponse); #B
rpc QueryMetrics(QueryMetricsRequest) returns
➥(QueryMetricsResponse); #B
rpc RecordSpan(RecordSpanRequest) returns
➥(RecordSpanResponse); #C
rpc RecordGeneration(RecordGenerationRequest)
returns (RecordGenerationResponse); #D
rpc GetTrace(GetTraceRequest) returns (Trace); #E
rpc QueryTraces(QueryTracesRequest)
➥returns (QueryTracesResponse); #E
rpc RecordScore(RecordScoreRequest)
➥returns (RecordScoreResponse); #F
rpc QueryScores(QueryScoresRequest)
➥returns (QueryScoresResponse); #F
rpc GetCostReport(CostReportRequest)
➥returns (CostReport); #G
rpc SetBudgetAlert(SetBudgetAlertRequest)
returns (SetBudgetAlertResponse); #G
rpc GetBudgetStatus(GetBudgetStatusRequest)
➥returns (BudgetStatus); #G
rpc GetServiceHealth(ServiceHealthRequest)
returns (ServiceHealthResponse); #H
}The Observability Service exposes monitoring through a gRPC contract for ingesting and querying logs, metrics, spans, generations, and scores. Sessions remain managed elsewhere and are linked through session_id. section 7.3


















































