PromptOps manages prompts as configuration artifacts with behavioral impact in LLM applications. A change to a system prompt or user prompt template can alter application semantics as significantly as a code change, so prompts need versioning, evaluation, deployment controls, monitoring, and revision. section 9.3.4
A PromptOps lifecycle includes:
- Version prompt templates so changes can be tracked and poor-performing versions rolled back. section 9.3.4
- Run automated evaluations against a benchmark golden set whenever a template changes. Regression suites can check format adherence, refusal behavior, citations, escalation, domain accuracy, and resistance to instruction override. section 9.3.4
- A/B test prompt variants with subsets of users, and log the prompt version, response, and feedback for monitoring and auditability. section 9.3.4
Releasing a prompt revision
A team changes a prompt template and runs its golden-set regression suite, including positive, negative, adversarial, and representative user inputs. It then compares variants with subsets of users and records the version used for each response. If the revision performs poorly, the team can roll back to the previous version. section 9.3.4
PromptOps is one context-focused operational discipline: RAGOps manages indexing and retrieval pipelines, while AgentOps manages tools, memory, state, and orchestration. The labels and ownership boundaries vary across organizations. The broader context-as-code approach also applies version control, testing, and rollback to assets such as state schemas, tool definitions, and retrieval configurations. section 9.3.2 section 9.3.4

