liveBook · Manning

LLMOps

As explained in Context Engineering

Operational practices for developing, deploying, monitoring, and maintaining systems based on large language models.

LLMOps applies DevOps practices to the development and production operation of large language models. It is most relevant when an organization controls model serving, customization, or gateway behavior. Its lifecycle adapts, serves, monitors, versions, and deploys a model for a production context. section 9.3.3

Unlike traditional MLOps for tabular data and predictive models, LLMOps must manage nondeterministic outputs, large parameter counts, and integration with external knowledge bases. Operational goals include performance, robustness, and compliance with governance constraints. section 9.3.3

A typical lifecycle starts with a pretrained foundation model and includes:

  • Data preparation and model customization to supply domain-specific behavior. section 9.3.3
  • Deployment and serving with the configuration and controls required by the production context. section 9.3.3
  • Monitoring and versioning, alongside operational components such as gateways and caches. section 9.3.3 section 9.3.2

Customizing a model for enterprise tasks

A team can fine-tune a general-purpose model on domain-specific data such as support tickets, legal documents, financial reports, or internal task demonstrations. Fine-tuning updates model weights and requires dataset governance, training infrastructure, validation, and rollback plans. Parameter-efficient fine-tuning reduces the number of parameters trained while keeping most base parameters unchanged. section 9.3.3

Figure 9.4 LLMOps lifecycle for context-aware systems, from pre-trained model to customization, caching, monitoring, versioning, and deployment
llmops v5

Follow the lifecycle from a pretrained model through customization, caching, monitoring, versioning, and deployment. section 9.3.3

The operating boundary depends on model ownership. Organizations hosting models generally need the broader lifecycle; API consumers still need provider selection, gateways, evaluation, and release controls, but usually do not own the full fine-tuning or runtime lifecycle. ContextOps and its specialized disciplines manage application context sources such as prompts, retrieval, tools, memory, and state. section 9.3.3 section 9.3.2

How Context Engineering covers it

How deep it goes
Outlines the lifecycle and customization options, including fine-tuning governance, validation, and rollback requirements.
Examples
Uses enterprise support tickets, legal documents, financial reports, and task demonstrations as potential fine-tuning data.
What you can do after
Evaluate operational responsibilities for hosted models versus API consumption and the requirements of fine-tuning.
How it connects
Relates DevOps, MLOps, and FMOps to model operations and distinguishes them from GenAIOps and ContextOps.

Also explained in

Compare how 7 books explain LLMOps →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Context Engineering. The section links point to the parts of the book it draws on.