LLMOps extends MLOps for production applications built with large language models. It preserves reliable infrastructure, deployment, monitoring, automation, and testing while adding controls for retrieval, prompts, safety, cost, and nondeterministic behavior. section 1.3.2 section 12.0
Production LLM systems add capabilities around the existing ML platform:
- Process and ingest documents for knowledge-grounded applications. section 1.3.2
- Use retrieval systems and vector databases for semantic search. section 1.3.2 section 12.0
- Manage prompts and test nondeterministic, multistep behavior. section 1.3.2 section 12.0
- Apply safety guardrails and optimize operational cost. section 1.3.2 section 13.4.6
DakkaBot documentation assistant
A RAG-powered documentation assistant combines document ingestion, semantic retrieval, and generative responses. Its production design retains existing Kubernetes, CI/CD, deployment, and monitoring foundations while adding LLM-specific prompts, testing, and safety controls. section 1.5 section 12.0
Figure 1.5 Traditional MLOps (left) extended with LLMOps components (right) for production LLM systems. Chapters 12 and 13 explore these extensions in detail.
Maps traditional MLOps components to the LLMOps additions needed for production LLM systems. section 1.3.2
Figure 12.1 The mental map where we’re focusing on building our RAG application (12)
Locates document ingestion, vector databases, prompt management, safety guardrails, and specialized testing within the production LLM architecture. section 12.0
