LLMOps applies MLOps practices to the development, deployment, monitoring, and maintenance of systems built around large language models. The label is useful because scaling introduces operational problems that rudimentary MLOps may not handle, although both disciplines share the same foundations. section 3.1 section 3.2
Production LLM systems extend familiar operations with specialized techniques:
- Use quantization, pruning, and knowledge distillation to reduce model size; quantization is especially practical because it can occur after training without fine-tuning. section 3.5
- Parallelize workflows across data, tensor, and pipeline dimensions: data parallelism increases throughput, tensor parallelism increases speed, and pipeline parallelism enables execution across available infrastructure. section 3.5
- Combine the three approaches as 3D parallelism to improve utilization by covering one another’s weaknesses. section 3.5
- Add vector databases when applications need rapid search and retrieval of embeddings. section 3.5
A practical LLMOps distinction
A platform may retain familiar MLOps infrastructure while adding model-compression methods, distributed execution, and vector-database retrieval. The shared foundation remains, but “good enough” operational choices can fail at LLM scale. section 3.1 section 3.5
