A token is a unit counted in model requests. Token metrics distinguish prompt tokens sent to a model, completion tokens it generates, and their total; providers use these counts for cost calculations. section 3.8.1
Request metrics can also record:
- Tokens served from response cache, which have zero cost, and tokens benefiting from prompt caching, which have reduced cost. section 3.8.1
Listing 3.16 RequestMetrics dataclass for observability
@dataclass
class RequestMetrics:
timestamp: datetime
provider: str #A
model: str
prompt_tokens: int
completion_tokens: int
total_tokens: int
duration_ms: float
time_to_first_token_ms: Optional[float] #B
success: bool
error_type: Optional[str]
cache_hit: bool
workflow_id: Optional[str] #C
user_id: Optional[str] #C
The request-metrics dataclass shows token counts alongside provider, model, caching, and workflow attribution fields. section 3.8.1







































