liveBook · Manning

Trace_operation

As explained in Designing AI Systems

trace_operation is a context-manager method that records a non-LLM service operation as a tracing span and yields context for nested calls.

trace_operation is a context manager for tracing non-LLM service work. It creates a span, records timing and errors, attaches operation-specific attributes, and yields context for nested calls. section 7.5.1

A TracedService uses an ObservabilityClient to start and end spans locally. Completed spans remain in an in-memory buffer until batched delivery through the RecordSpan RPC, keeping telemetry off the request's critical path. section 7.5.1

Use trace_operation for service work such as:

The method receives an operation name, an incoming TraceContext, and keyword attributes. It starts a span under the caller's span, then yields a child context carrying the same trace ID and the new span ID. Downstream services pass that context forward to preserve the trace tree. section 7.5.1

Nested service tracing

A Guardrails Service can wrap validation in trace_operation and pass the yielded context to a Model Service. The Model Service can then create a trace_generation child beneath the validation span, while the workflow caller remains unaware of the instrumentation. section 7.5.1 section 7.5

Listing 7.6 TracedService base class with span and generation context managers
class TracedService:
    """Base class providing automatic tracing for all
    platform services."""
 
    def __init__(self, service_name: str,
                 observability: ObservabilityClient):
        self.service_name = service_name
        self.observability = observability
 
    @contextmanager
    def trace_operation(self, operation: str,
                         trace_context: TraceContext,
                         **attributes):
        span = self.observability.start_span(    #A
            trace_id=trace_context.trace_id,
            parent_span_id=trace_context.span_id,
            service=self.service_name,
            operation=f"{self.service_name}.{operation}",
            attributes=attributes)
        child_context = TraceContext(    #B
            trace_id=trace_context.trace_id,
            span_id=span.span_id,
            parent_span_id=trace_context.span_id,
            workflow_id=trace_context.workflow_id)
        try:
            yield child_context    #C
            self.observability.end_span(
                span.span_id, 
➥status=StatusCode.OK)    #D
        except Exception as e:
            self.observability.end_span(
                span.span_id,
                status=StatusCode.ERROR,
                error_message=str(e))
            raise
 
    @contextmanager
    def trace_generation(self, trace_context: TraceContext,
                          model: str, 
➥**attributes):    #E
        gen = self.observability.start_generation(
            trace_id=trace_context.trace_id,
            parent_span_id=trace_context.span_id,
            service=self.service_name,
            model=model, **attributes)
        try:
            yield gen
            self.observability.end_generation(gen.span_id)
        except Exception as e:
            self.observability.end_generation(
                gen.span_id,
                status=StatusCode.ERROR,
                error_message=str(e))
            raise

Inspect the TracedService implementation to see service naming, local span creation, child-context construction, and operation-specific attributes. section 7.5.1

How Designing AI Systems covers it

How deep it goes
Explains span creation, child-context propagation, local buffering, timing, errors, and flushing, while deferring buffering details.
Examples
Uses session retrieval, knowledge-base search, guardrail validation, exchange storage, and nested Model Service classification checks.
What it assumes
Assumes TraceContext and ObservabilityClient interfaces, while TracedService supplies the context-manager instrumentation.
What you can do after
Apply it via TracedService to instrument service boundaries and preserve parent-child trace relationships without wrapping workflow calls manually.
How it connects
Connects distributed tracing with TracedService, TraceContext, ObservabilityClient, spans, trace_generation, and gRPC delivery.

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.