Prompts can carry business logic: they tell a language model how to use retrieved context, answer a question, and handle uncertainty. Hardcoding them as application strings ties every revision to a code deployment. Prompt management instead makes creation, testing, iteration, logging, versioning, and deployment a controlled workflow. section 13.1 section 4.1.3
A production workflow with Langfuse separates the prompt asset from the application that uses it:
- Create a named chat prompt with system instructions and runtime variables such as {{context}} and {{query}}. Keep its structure visible rather than assembling it through application-side string concatenation. section 13.1.2
- Assign a deployment label such as production and store model settings in prompt configuration. Labels let multiple versions serve different environments without changing application code. section 13.1.2
- At request time, fetch the prompt by name and label, apply its configuration, compile it with the retrieved context and user query, and record its version with the response. section 13.1.2
- Test variants against real traffic with A/B routing and traces. Compare response quality, user satisfaction, and business metrics before promoting a change; retain version history so changes can be rolled back. section 13.1.3 section 13.1.2
Listing 13.1 Creating managed prompts in Langfuse with versioning
langfuse.create_prompt(
name="dakkabot-system", #1
type="chat", #2
prompt=[
{
"role": "system",
"content": """You are DakkaBot, DataKrypt's expert
developer assistant. Based on the provided DataKrypt documentation,
answer the user's question accurately and helpfully.
**Documentation Context:**
{{context}}
**User Question:** {{query}}
**Instructions:**
- Provide accurate, detailed answers based on the documentation
- Include specific steps, code examples, or configurations
when relevant
- If the question requires multiple services or
procedures, explain the complete workflow
- Use proper formatting for code blocks and lists
- If you're not sure about something, say so rather than guessing
- Stay focused on DataKrypt-specific information
**Answer:**"""
}
],
labels=["production"], #3
config={
"model": "gemini-2.5-flash",
"temperature": 0.0,
"max_tokens": 1000
}, #4
tags=["dakkabot", "rag", "documentation"] #5
)
The managed chat-prompt listing shows a stable name, message role, documentation instructions, and runtime placeholders—the pieces that move out of hardcoded application strings. section 13.1.2
Listing 13.6 Template compilation and tracked response generation
class DakkaBotCore:
async def process_query(self, query: str) -> Dict[str, Any]:
try:
docs = await self.retrieve_documents(query)
context = self.format_context(docs)
prompt_obj = self.langfuse.get_prompt(
"dakkabot-system",
type="chat",
label="production"
)
prompt_config = prompt_obj.config
self.llm.model = prompt_config.get("model", "gemini-2.5-flash")
self.llm.temperature = prompt_config.get("temperature", 0.0)
compiled_messages = prompt_obj.compile( #1
context=context, query=query) #1
metadata = {
"prompt_name": "dakkabot-system",
"prompt_version": prompt_obj.version #2
}
response = self.llm.invoke(
compiled_messages,
config={
"callbacks": [self.langfuse_handler],
"metadata": metadata
}
)
return {
"response": response.content,
"sources": docs,
"prompt_version": prompt_obj.version
}
The request-time listing connects retrieval to prompt compilation and attaches the selected prompt version to generation metadata. section 13.1.2
Documentation assistant with a managed boundary
A documentation assistant retrieves relevant documents, formats them as context, and compiles its production-labeled system prompt with that context and the question. A separate managed text prompt supplies a redirect with a topic and contact channel when processing fails. The returned result can identify the prompt version or indicate that the fallback was used. section 13.1.2
Remote prompt storage introduces an availability and latency concern. Langfuse caches prompts after the first fetch, refreshes them in the background, and supports fallback prompts and prefetching for network problems. Start by migrating one prompt, then test updates and failure behavior before relying on remote management across an entire RAG application. section 13.1.3
