An LLM application may need more than one model call to answer a query. In a retrieval-augmented generation (RAG) system, a query can trigger document retrieval, context assembly, response generation, and output validation. These operations form a multi-step reasoning chain: an orchestrated path from the user’s question to the final response. Unlike a simple data-in, prediction-out workflow, a failure anywhere along that path can degrade the answer. section 12.1.1 section 13.4.6
Make the path observable so a poor answer can be traced to a stage rather than treated as one opaque model failure:
- Inspect which documents retrieval returned and which of them influenced the response. section 12.4
- Inspect how context and the prompt were assembled before generation, then compare them with the model’s output. section 12.1.1 section 12.4
- Track timing and token usage by stage; check whether an error began in retrieval, context assembly, or generation. section 12.4.4
Tracing a documentation answer
Suppose a documentation assistant returns an irrelevant answer. A trace of its retrieval, augmentation, and generation stages can help distinguish irrelevant retrieved documents from poorly assembled context or a generation failure. Langfuse span tracking can show the query’s path, with timing and token usage at each step. This is visibility into the application’s operations, not a claim that a trace reveals every internal decision of the model. section 12.0 section 12.4.4
