A generation is a specialized span for a model call. It preserves the timing, status, attributes, and trace relationships of a span while adding model-specific fields. Keeping those fields first-class lets operators query model behavior directly instead of scanning generic spans for key-value attributes. section 7.2.1 section 7.4.3
Listing 7.5 Span, Generation, and Trace Protocol Buffer definitions
message Span {
string trace_id = 1; #A
string span_id = 2;
string parent_span_id = 3;
string service = 4;
string operation = 5; #B
google.protobuf.Timestamp start_time = 6;
google.protobuf.Timestamp end_time = 7;
SpanStatus status = 8;
string error_message = 9;
map<string, string> attributes = 10;
map<string, double> numeric_attributes = 11;
repeated SpanEvent events = 12;
}
message Generation {
Span span = 1; #C
string model = 2;
string provider = 3;
string requested_model = 4;
int32 prompt_tokens = 5;
int32 completion_tokens = 6;
double cost_usd = 7;
bool cache_hit = 8;
bool fallback_used = 9;
double time_to_first_token_ms = 10;
}
message Trace {
string trace_id = 1;
string session_id = 2; #D
repeated Span spans = 3;
repeated Generation generations = 4;
string input = 5;
string output = 6;
double total_duration_ms = 7;
double total_cost_usd = 8;
int32 total_tokens = 9;
repeated Score scores = 10; #E
repeated string tags = 11;
}The Generation message wraps a Span and adds the fields needed to track model usage and performance. section 7.4.3
The model-specific fields record:
- The model, provider, and requested model. section 7.4.3
- Prompt and completion token counts, plus cost in USD. section 7.4.3
- Cache-hit and fallback status, plus time to first token in milliseconds. section 7.4.3
A generation belongs in the same trace tree as other work. Services propagate trace context, and the observability service correlates recorded spans and generations by trace_id to assemble the request. Scores can attach to a generation to capture quality signals separately from execution telemetry. section 7.4.3 section 7.3.1
A model call within a request
A patient intake request records a GPT-4o generation with 3,100 prompt tokens, 180 completion tokens, $0.018 cost, 340 ms to first token, and no cache hit. Retrieval, validation, and persistence are recorded as regular spans in the same trace. section 7.2.2
Instrument model calls with a generation context manager rather than the generic operation wrapper. It creates the record, captures timing and errors, and provides a child context for nesting. Local start and end methods buffer telemetry; completed generations are sent in batches through RecordGeneration. The separate ingestion path supports a generation-specific index for queries such as calls above a cost threshold. section 7.5.1 section 7.3.1
Listing 7.7 Model Service chat method with automatic generation tracking
class ModelService(TracedService):
def chat(self, request, trace_context):
with self.trace_generation( #A
trace_context, model=request.model,
) as gen_ctx:
provider = self._select_provider
➥(request) #B
response = provider.chat(
request.messages, request.config)
gen_ctx.update( #C
prompt_tokens=response.usage.prompt_tokens,
completion_tokens=response.usage.completion_tokens,
cost_usd=self._calculate_cost(response),
)
return responseThe chat implementation opens a generation context and updates it with token usage and calculated cost after the provider responds. section 7.5.1
