Token count
A token count is a measure of the number of tokens present in text or processed or generated during a model or corpus operation.
Build AI-Enhanced Web Apps
Token usage is the quantity of input and output tokens consumed by a language-model interaction.
- How it is used
- Tracks LLM consumption to manage costs across document-summarization calls and keep conversation histories within token limits.
- How deep it goes
- Shows callback-based usage tracking and outlines history counting, but does not implement a tokenizer or calculate prices.
- Examples
- Uses a LangChain.js `TokenTracker` callback, a `streamText` output cap, and a rolling ten-message history.
- What you can do after
- Operate/configure token monitoring to inspect per-call usage and adjust output or retained history against limits.
Spring AI in Action
Token usage is the measured quantity of input and output tokens consumed during a language-model interaction.
- How it is used
- Tracks language-model consumption to estimate billing, control prompt costs, and observe Spring AI operations.
- How deep it goes
- Explains input, output, and total counts, metric tags, Java logging, and Actuator queries, but not tokenization internals.
- Examples
- Demonstrates board-game prompts, ChatResponse logging, and filtered Actuator measurements for input, output, total, model, and operation.
- What you can do after
- Diagnose token spending by logging ChatResponse metadata and querying gen_ai.client.token.usage with filtering tags.
Machine Learning Platform Engineering
Token usage is the quantity of language-model input and output tokens consumed during an interaction or application operation.
- How it is used
- Monitors LLM resource consumption for efficiency, observability, performance analysis, and operational cost control.
- How deep it goes
- Explains input/output ratios, reasoning-step allocation, prompt reduction, tracing, alerts, and budgets without specifying tokenizer internals.
- Examples
- Uses verbose and concise prompts, Langfuse traces, RAG pipelines, token ratios, cost dashboards, alerts, and budget thresholds.
- What you can do after
- Diagnose/debug with Langfuse by tracing token usage, locating expensive operations, and validating prompt reductions against quality and safety.
Read how Machine Learning Platform Engineering explains token count →
How Large Language Models Work
The quantity of tokens consumed as input to or produced as output by a language model.
- How it is used
- Tracks model input and output to explain cost, latency, capacity, and language representation.
- How deep it goes
- Explains BPE-driven counting with a worked comparison, but stops short of formal tokenizer algorithms.
- Examples
- Uses GPT-3 tokenizations of “I’m running” and “I’m runnin,” plus cross-language cost comparisons.
- What you can do after
- Evaluate tokenization-related cost, latency, language disparities, and software differences before deploying or upgrading systems.
Read how How Large Language Models Work explains token count →
Related concepts
- tiktokenmeasured by · tiktoken is an OpenAI-developed library that efficiently encodes text using model-specific byte-pair-encoding tokenizers and supports token counting.
- prompt engineeringused by · Prompt engineering is the deliberate design and refinement of instructions, context, examples, or other inputs to guide an AI model’s behavior or outputs.
- frequency analysisused by · Frequency analysis examines how often symbols, values, events, or linguistic units occur in data, text, or signals.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.