trace_generation is a TracedService context manager for instrumenting language-model calls. It creates a generation trace primitive, records model identity, timing, errors, token counts, cost, and time to first token, then yields a child context for nested operations. section 7.5.1
The operation keeps tracing off the request's critical path. start_generation() and end_generation() write locally to an in-memory ObservabilityClient buffer; later batching flushes completed generations through RecordGeneration. section 7.5.1
A typical generation-tracing flow:
- Enter trace_generation with the incoming TraceContext and model name. section 7.5.1
- Close the context so timing and errors are finalized while preserving parent-child trace relationships. section 7.5.1
ModelService.chat
A ModelService method wraps provider execution with `with self.trace_generation(trace_context, model=request.model) as gen_ctx:`. After receiving the response, it calls `gen_ctx.update(...)` with token usage and calculated cost before returning the response.
