Token count measures the language-model input and output tokens consumed by an interaction or application operation. In production, it links prompt design, context length, response detail, cost, and performance. section 12.4.4 section 13.4.4
Track token count to make LLM behavior operationally visible:
- Separate input and output tokens; their ratio indicates whether prompts and assembled context are efficient. section 12.4.4 section 12.4.3
- Monitor token allocation across retrieval, context assembly, and generation so inefficient reasoning paths become identifiable. section 12.4.4
- Relate usage to query complexity, context length, response detail, model choice, and per-query cost rather than assuming stable consumption. section 12.4.4 section 13.4.5
- Set aggregate and per-query budgets, alert before thresholds, and inspect expensive outliers for inefficient prompts or abuse. section 13.4.5
Reducing prompt overhead
A verbose assistant prompt can be replaced with a concise identity, documentation instruction, context placeholder, and Q:/A: structure. The optimized version achieves the same functional outcome with 70% fewer tokens, subject to quality and safety validation through iterative refinement and A/B testing. section 13.4.4
Listing 12.19 Langfuse observability integration and tracing
class DakkaBotCore:
def __init__(self):
... #1
self.langfuse_handler = None #2
async def initialize(self):
try:
...
self.llm = ...
langfuse = Langfuse(
public_key=os.environ[ #4
"LANGFUSE_PUBLIC_KEY"], #4
secret_key=os.environ[ #4
"LANGFUSE_SECRET_KEY"], #4
host=os.environ.get( #4
"LANGFUSE_HOST", #4
"http://localhost:3000") #4
) #4
self.langfuse_handler = CallbackHandler() #5
self.initialized = True
except Exception as e:
raise RuntimeError(f"Failed to initialize DakkaBot: {str(e)}")
async def process_query(self, query: str) -> Dict[str, Any]:
try:
docs = ...
context = ...
prompt = ...
response = self.llm.invoke(
prompt,
config={
"callbacks": [self.langfuse_handler], #6
"metadata": {
"query": query, #7
"doc_count": len(docs), #7
"context_length": len(context), #7
"user_session": "demo" #7
}
}
)
return {
"response": response.content,
"sources": [...], # Source processing
"doc_count": len(docs),
"error": None
}
except Exception as e:
return {
"response": "Sorry! Please try again!",
"sources": [],
"doc_count": 0,
"error": str(e)
}
Inspect a Langfuse callback integration that automatically captures prompts, responses, token usage, latency, retrieval activity, and pipeline traces. section 12.4.3
concise_prompt = """
You are DakkaBot. Answer based on this DataKrypt documentation:
{context}
Q: {query}
A:
"""
Compare a concise production prompt with its verbose counterpart and identify reductions in identity, instruction, formatting, and redundancy overhead. section 13.4.4
