liveBook · Manning

Mlops

As explained in LLMs in Production

Machine learning operations is the discipline of reliably deploying, monitoring, maintaining, and governing machine-learning systems throughout their life cycle.

MLOps is the practical discipline of reliably deploying and maintaining machine-learning systems in production. It spans the full life cycle: acquiring data, training models, monitoring behavior, and eventually terminating systems. Reliable operation depends on orchestration, versioning, feedback loops, CI/CD, security, resource provisioning, and data governance. section 3.1

Infrastructure is the how; the machine-learning life cycle is the why. A small system may run from one data scientist’s laptop, but large language models require sturdier infrastructure because models may exceed local storage or GPU capacity. LLMOps therefore extends MLOps challenges rather than clearly replacing MLOps as a separate discipline. section 3.1 section 3.4

A production-oriented MLOps platform commonly combines:

  • Data stores for storing, governing, and securing data, ranging from databases and warehouses to lakes and meshes. section 3.4.1
  • Orchestrators such as Airflow, Prefect, and Mage for automating multistep workflows across resources and services. section 3.4.1
  • Pipelines for moving and cleaning data, scheduling training jobs, running batch inference, and loading results. section 3.4.1
  • Container registries for consistent execution, with streaming integrations added when processing needs require them. section 3.4.1
  • Compression and distributed computing for making oversized models smaller and executable across multiple resources. section 3.3

From laptop prototype to production platform

A scikit-learn model running on one data scientist’s laptop can work as a scrappy prototype. An LLM-oriented platform must instead coordinate data stores, pipelines, containers, GPUs, and other infrastructure around the model life cycle. section 3.4

Figure 3.7 A high level view of an MLOps infrastructure with LLMs in mind. This diagram attempts to cover the full picture and the complexity of the many tools involved to make ML models work in production.
figure

Use the infrastructure overview to inspect how data stores, orchestrators, pipelines, containers, and model-serving components fit together in an LLM-oriented MLOps system. section 3.4

How LLMs in Production covers it

How deep it goes
Explains lifecycle principles and infrastructure layers, then names practices and tools without detailing implementation procedures.
Examples
Uses a laptop-based scikit-learn system, Airflow, Prefect, Mage, data pipelines, containers, GPUs, and an LLM infrastructure diagram.
What it assumes
Builds from DevOps and DataOps foundations, including data stores, orchestrators, pipelines, containers, and streaming integrations.
What you can do after
Evaluate production needs by separating the machine-learning life cycle from the infrastructure required to support it.
How it connects
Connects MLOps to LLMOps, DataOps, DevOps, workflow orchestration, CI/CD, data governance, distributed computing, and model monitoring.
Cover of LLMs in Production
liveBookmanning.com
Read LLMs in Production on liveBook Get the book at manning.com

Also explained in

Compare how 7 books explain mlops →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.