Langfuse is an open-source platform for observing LLM applications and managing prompts. In a retrieval-augmented generation (RAG) system, infrastructure metrics can show that a request was slow, but not which documents were retrieved or why an answer was poor. Langfuse traces retrieval and generation alongside latency, token usage, and estimated cost. section 12.1.3 section 12.5
A local integration has three steps:
- Start the Langfuse stack with Docker Compose, then open http://localhost:3000. Create an account, organization, and project, and generate API keys. section 12.4.1
- Configure the application with LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, and, when needed, LANGFUSE_HOST. section 12.4.3
- Attach a Langfuse callback handler to LangChain execution. It captures LLM calls, retrieval steps, timings, errors, token usage, and estimated costs without instrumenting each pipeline step separately. section 12.1.3 section 12.4.2 section 12.4.3
% git clone https://github.com/langfuse/langfuse.git % cd langfuse % docker compose up
The Docker Compose commands provide the local deployment used before configuring the dashboard and API keys. section 12.4.1
from langchain.callbacks import LangfuseCallbackHandler
langfuse_handler = LangfuseCallbackHandler()
result = chain.invoke(
{"query": "How do I rotate API keys?"},
config={"callbacks": [langfuse_handler]}
)
The callback example shows where tracing enters a LangChain RAG query. section 12.1.3
For production diagnosis, enrich traces with query, retrieval, context-use, and session metadata. The dashboard then connects inputs and outputs to intermediate operations, latency, and cost. Teams can investigate failed calls, weak retrieval, performance degradation, or query types that consume disproportionate resources; a trace provides evidence for investigation, not an automatic judgment that an answer is correct. section 12.4.3 section 13.4.5
Langfuse also keeps prompts outside application code. Register a named text or chat prompt with template variables, configuration, and a deployment label such as production. At request time, fetch that label, compile the prompt with the retrieved context and user query, and record its version with the response. Reassigning labels can promote or roll back prompt versions without changing application code. section 13.1.2 section 13.1.3
Listing 13.5 Dynamic prompt fetching and configuration
class DakkaBotCore:
async def process_query(self, query: str) -> Dict[str, Any]:
try:
docs = ...
context = ...
prompt_obj = self.langfuse.get_prompt(
"dakkabot-system", #1
type="chat",
label="production" #2
)
prompt_config = prompt_obj.config #3
self.llm.model = prompt_config.get( #3
"model", "gemini-2.5-flash") #3
self.llm.temperature = prompt_config.get( #3
"temperature", 0.0) #3
The dynamic-fetch listing shows how an application selects a production-labeled prompt and reads its model configuration. section 13.1.2
Use trace and cost trends to compare prompt versions and investigate expensive queries. Langfuse supports prompt-variant testing, while semantic evaluation and safety enforcement remain distinct parts of a production LLM system. section 13.1.3 section 13.4.5 section 13.0
