ObservabilityClient buffers spans, logs, and metrics locally and delivers them to the Observability Service in batches. Each platform service holds a client, separating telemetry publication from network delivery so individual events do not require a request-path round-trip. section 7.5.3 section 7.8
TracedService uses local client methods to start and end spans and generations. The context managers record timing and errors and propagate child contexts; completed spans and generations are later delivered through RecordSpan and RecordGeneration. Service-specific attributes make spans queryable. section 7.5.1
Listing 7.6 TracedService base class with span and generation context managers
class TracedService:
"""Base class providing automatic tracing for all
platform services."""
def __init__(self, service_name: str,
observability: ObservabilityClient):
self.service_name = service_name
self.observability = observability
@contextmanager
def trace_operation(self, operation: str,
trace_context: TraceContext,
**attributes):
span = self.observability.start_span( #A
trace_id=trace_context.trace_id,
parent_span_id=trace_context.span_id,
service=self.service_name,
operation=f"{self.service_name}.{operation}",
attributes=attributes)
child_context = TraceContext( #B
trace_id=trace_context.trace_id,
span_id=span.span_id,
parent_span_id=trace_context.span_id,
workflow_id=trace_context.workflow_id)
try:
yield child_context #C
self.observability.end_span(
span.span_id,
➥status=StatusCode.OK) #D
except Exception as e:
self.observability.end_span(
span.span_id,
status=StatusCode.ERROR,
error_message=str(e))
raise
@contextmanager
def trace_generation(self, trace_context: TraceContext,
model: str,
➥**attributes): #E
gen = self.observability.start_generation(
trace_id=trace_context.trace_id,
parent_span_id=trace_context.span_id,
service=self.service_name,
model=model, **attributes)
try:
yield gen
self.observability.end_generation(gen.span_id)
except Exception as e:
self.observability.end_generation(
gen.span_id,
status=StatusCode.ERROR,
error_message=str(e))
raiseSee how the tracing context manager uses the client and constructs a child trace context. section 7.5.1
The delivery mechanism balances network overhead, timeliness, and bounded memory:
- Finalizing a span adds it to an in-memory buffer; reaching the configured batch size triggers a flush of up to that many spans. section 7.5.3
- A failed span batch returns to the front of the buffer for a later retry. If the buffer exceeds its maximum size, the oldest data is dropped. section 7.5.3
- A background timer flushes spans, metrics, and logs even when buffers are not full, supporting low-traffic delivery and retries. section 7.5.3
Listing 7.9 ObservabilityClient batching and buffering
class ObservabilityClient:
def __init__(self, channel,
batch_size: int = 100,
flush_interval_seconds: float = 5.0,
max_buffer_size: int = 10_000):
self._stub = ObservabilityServiceStub(channel)
self._span_buffer: list[Span] = [] #A
self._metric_buffer: list[MetricRecord] = [] #A
self._log_buffer: list[LogEvent] = [] #A
self._batch_size = batch_size
self._max_buffer_size = max_buffer_size
self._flush_interval = flush_interval_seconds
self._flush_thread = threading.Thread( #B
target=self._periodic_flush, daemon=True) #B
self._flush_thread.start()
def end_span(self, span_id: str,
status: StatusCode = StatusCode.OK,
error_message: str = None):
span = self._finalize_span(
span_id, status, error_message)
self._span_buffer.append(span)
if len(self._span_buffer) >= self._batch_size:
self._flush_spans() #C
def _flush_spans(self):
if not self._span_buffer:
return
batch = self._span_buffer[:self._batch_size]
self._span_buffer = (
self._span_buffer[self._batch_size:])
try:
self._stub.RecordSpan( #D
RecordSpanRequest(spans=batch)) #D
except Exception:
if (len(self._span_buffer)
< self._max_buffer_size):
self._span_buffer = ( #E
batch + self._span_buffer) #E
# else: drop oldest data to prevent
# unbounded memory growth
def _periodic_flush(self):
while True:
time.sleep(self._flush_interval)
self._flush_spans() #F
self._flush_metrics() #F
self._flush_logs() #FInspect the client’s buffers, delivery configuration, and background flush mechanism. section 7.5.3
Publishing model metrics
A Model Service publisher translates request metrics into a request counter, a cost counter, and a duration histogram. Provider, model, workflow ID, and cache-hit labels make the resulting metrics queryable by dimension. section 7.5.2
Automatic instrumentation covers platform service operations. For application logic outside those services, developers can add a custom span through the SDK, publish custom metrics through the client, and emit structured logs referencing the current span ID. section 7.5.4
