LLMOps
LLMOps is the set of engineering and operational practices for developing, deploying, evaluating, monitoring, and maintaining large language model systems and applications.
Generative AI in Action
LLMOps is the set of operational practices for developing, deploying, monitoring, governing, and maintaining large language models and applications built with them.
- How it is used
- Guides the transition of LLM applications from proof of concept to reliable, secure production operation.
- How deep it goes
- Defines the lifecycle, compares it with MLOps, and supplies deployment checklist items rather than an end-to-end implementation.
- Examples
- Uses enterprise data-center hosting and a production checklist covering scaling, rate limits, observability, and security.
- What you can do after
- Evaluate/understand production readiness by checking an LLM application's scaling, monitoring, security, and governance needs.
Financial AI in Practice
LLMOps is the set of practices for developing, deploying, monitoring, and maintaining systems based on large language models.
- How it is used
- Positions LLMOps as management of fraud RAG pipelines and a cross-training path for data engineers.
- How deep it goes
- Details fraud RAG operational components and human oversight, but supplies no executable implementation or step-by-step deployment procedure.
- Examples
- Uses a human-reviewed moderation workflow and fraud case summaries grounded in policies, case notes, and investigation documents.
- What you can do after
- Evaluate/understand it by identifying operational controls for fraud RAG and the need to preserve human decisions in moderation.
Machine Learning Platform Engineering
LLMOps is the set of engineering and operational practices for developing, deploying, monitoring, and managing large language model applications.
- How it is used
- Extends MLOps practices to develop, deploy, monitor, secure, and manage production LLM applications.
- How deep it goes
- Explains architecture and operational concerns through RAG systems, while deferring incident response, compliance, scaling, drift, and business metrics.
- Examples
- Uses DakkaBot, a RAG-powered documentation assistant, plus document ingestion, semantic search, prompt management, and specialized testing.
- What you can do after
- Build and operate production-ready RAG applications by extending existing ML platforms with LLM-specific retrieval, prompting, testing, and safety components.
Read how Machine Learning Platform Engineering explains LLMOps →
The Art of AI Product Development
LLMOps is the set of operational practices and automation specialized for deploying, monitoring, maintaining, and updating large language model systems.
- How it is used
- Structures language-model deployment and improvement as an automated lifecycle of monitoring, evaluation, error analysis, and updates.
- How deep it goes
- Details prompt versioning, rollback, evaluation, compression, and tuning, but does not provide implementation code or operational runbooks.
- Examples
- Uses prompt version control, graphical collaboration tools, prompt compression, automated tuning, and custom evaluation metrics.
- What you can do after
- Operate language-model workflows by automating monitoring, evaluation, error tracking, prompt management, and model updates.
Read how The Art of AI Product Development explains LLMOps →
Building Reliable AI Systems
LLMOps is the engineering discipline of managing large language models in production, including their deployment, evaluation, and ongoing operation.
- How it is used
- Frames the infrastructure needed to keep LLM applications reliable, safe, and cost-aware after deployment.
- How deep it goes
- Explains a five-layer architecture and operational trade-offs, but does not specify implementation code in the supplied passages.
- Examples
- Uses a support assistant that passed internal tests before producing unpredictable results and a $47,000 monthly bill.
- What you can do after
- Operate/configure it with model routing, quality monitoring, cost alerts, and continuous improvement processes.
LLMs in Production
LLMOps is the discipline of developing, deploying, monitoring, and maintaining applications and systems built around large language models.
- How it is used
- Organizes the reliable deployment and maintenance of large-language-model systems at production scale.
- How deep it goes
- Extends MLOps foundations into compression, parallelism, and retrieval infrastructure, while emphasizing that scale introduces additional caveats.
- Examples
- Uses quantization, pruning, knowledge distillation, three-way parallelism, and vector-database embedding retrieval.
- What you can do after
- Apply compression, parallelism, and retrieval infrastructure when operating LLM workflows beyond basic MLOps capabilities.
Related concepts
- mlopskind of · MLOps is the discipline of applying engineering and operational practices to develop, deploy, monitor, maintain, and manage machine-learning systems throughout their life cycle.
- DevOpsuses · DevOps is a set of organizational, cultural, and technical practices that integrates software development and IT operations through collaboration, automation, and feedback.
- language modeluses · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
- evaluationhas part · Evaluation is the systematic assessment of a system, result, or approach against relevant criteria, evidence, or objectives.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.