liveBook · Manning

Trace tree

As explained in Designing AI Systems

A span tree is the hierarchical structure that represents nesting and parent-child relationships among spans in a trace.

A trace tree is the hierarchical structure that records how operations and model generations nest across services. Each span carries a trace ID and span ID; child contexts forward the current span as the parent for downstream work. section 7.5.1

A production tracing path separates instrumentation from transport:

  • Create a local span or generation through an observability client, recording timing, errors, service, operation, and attributes. section 7.5.1
  • Yield a child context whose parent_span_id points to the current operation, then pass it into nested service calls. section 7.5.1
  • Finalize completed records locally and batch them to the observability service through RecordSpan or RecordGeneration. section 7.5.1

Tracing application-specific work

A custom reranking step can join the platform trace by opening trace_operation("custom_rerank", trace_context, num_candidates=len(candidates), rerank_model="cross-encoder"). Its span inherits the trace ID and nests correctly beside platform-generated spans. section 7.5.4

Listing 7.6 TracedService base class with span and generation context managers
class TracedService:
    """Base class providing automatic tracing for all
    platform services."""
 
    def __init__(self, service_name: str,
                 observability: ObservabilityClient):
        self.service_name = service_name
        self.observability = observability
 
    @contextmanager
    def trace_operation(self, operation: str,
                         trace_context: TraceContext,
                         **attributes):
        span = self.observability.start_span(    #A
            trace_id=trace_context.trace_id,
            parent_span_id=trace_context.span_id,
            service=self.service_name,
            operation=f"{self.service_name}.{operation}",
            attributes=attributes)
        child_context = TraceContext(    #B
            trace_id=trace_context.trace_id,
            span_id=span.span_id,
            parent_span_id=trace_context.span_id,
            workflow_id=trace_context.workflow_id)
        try:
            yield child_context    #C
            self.observability.end_span(
                span.span_id, 
➥status=StatusCode.OK)    #D
        except Exception as e:
            self.observability.end_span(
                span.span_id,
                status=StatusCode.ERROR,
                error_message=str(e))
            raise
 
    @contextmanager
    def trace_generation(self, trace_context: TraceContext,
                          model: str, 
➥**attributes):    #E
        gen = self.observability.start_generation(
            trace_id=trace_context.trace_id,
            parent_span_id=trace_context.span_id,
            service=self.service_name,
            model=model, **attributes)
        try:
            yield gen
            self.observability.end_generation(gen.span_id)
        except Exception as e:
            self.observability.end_generation(
                gen.span_id,
                status=StatusCode.ERROR,
                error_message=str(e))
            raise

Inspect the TracedService context managers that create spans, construct child contexts, and preserve nesting across service boundaries. section 7.5.1

How Designing AI Systems covers it

How deep it goes
Defines span nesting, child-context propagation, local buffering, and batched export, while deferring buffering details.
Examples
Uses Guardrails Service nesting, Model Service generation tracking, and custom reranking spans as concrete tracing examples.
What it assumes
Assumes familiarity with service calls and observability clients while demonstrating context propagation from the instrumentation boundary.
What you can do after
Build it from scratch by propagating trace contexts, opening nested spans, recording attributes, and exporting completed records.
How it connects
Connects distributed tracing with spans, context managers, observability clients, generation tracking, and gRPC export.

Also explained in

Compare how 2 books explain trace tree →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Designing AI Systems. The section links point to the parts of the book it draws on.