liveBook · Manning

Input/output token ratio

As explained in Machine Learning Platform Engineering

An input/output token ratio compares the number of input tokens processed with the number of output tokens generated.

An input/output token ratio compares the tokens processed as input with the tokens generated as output. In production LLM applications, it acts as an efficiency indicator for prompt engineering and context assembly. section 12.4.4

Use the ratio alongside other production signals:

  • Relate token consumption to query complexity, context length, and response detail, which can vary unpredictably. section 12.4.4
  • Compare it with context utilization, cost per query type, and token-waste detection to identify inefficient context or unnecessarily verbose responses. section 12.4.4
  • Inspect token usage across retrieval, augmentation, and generation stages through detailed observability spans. section 12.4.4

How Machine Learning Platform Engineering covers it

How deep it goes
Defines the ratio and places it among related production metrics without specifying a calculation or threshold.
Examples
Uses Langfuse span tracking to inspect token usage through retrieval, augmentation, and generation stages.
What you can do after
Optimize prompt engineering and context assembly by locating wasteful input context or unnecessarily verbose output.
How it connects
Connects token efficiency with prompt engineering, context assembly, context utilization, observability, and cost analysis.
Cover of Machine Learning Platform Engineering
liveBookmanning.com
Read Machine Learning Platform Engineering on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Machine Learning Platform Engineering. The section links point to the parts of the book it draws on.