MLOps applies DevOps principles to develop, deploy, and manage machine-learning models and applications. It streamlines the lifecycle from data preparation and experimentation through training and serving while emphasizing quality, reliability, and scalability. section 11.6
MLOps operationalizes machine-learning systems by:
- Automating and orchestrating the machine-learning pipeline. section 11.6
- Maintaining reproducibility, traceability, and versioning for data, code, models, and experiments. section 11.6
- Monitoring and optimizing production performance, availability, and resource utilization. section 11.0 section 11.6
- Applying security, privacy, compliance, feedback, and continuous-improvement practices. section 11.6
Table 11.4 Differences between MLOps and LLMOps
|
Area
|
Traditional MLOps
|
LLMOps
|
|---|---|---|
|
Target audience
|
ML engineers, data scientists
|
Application developers, ML engineering, and data scientists
|
|
Components
|
Model, data, inference environments, features
|
LLMs, prompts, tokens, generations, APIs, embeddings, vector databases
|
|
Metrics
|
Accuracy (F1 score, precision, recall, etc.)
|
Quality (similarity), groundedness (accuracy), cost (tokens), latency, evaluations (Perplexity, BLEU, ROUGE, etc.)
|
|
Models
|
Typically built from scratch
|
Typically, prebuilt with inference via an API and multiple versions in production simultaneously
|
|
Ethical concerns
|
Bias in training data
|
Misuse and generation of harmful, fake, and biased output
|
Compare traditional MLOps with LLMOps across audiences, components, metrics, model delivery, and ethical concerns. section 11.6
