Infrastructure is the hardware, software, networking, storage, and operational machinery that makes computing systems usable in production. For machine learning, it is the how; the machine learning life cycle is the why. A laptop may support a scrappy scikit-learn system, but LLMs can outgrow local storage before serving begins. section 3.4
A practical infrastructure view includes:
- Data stores, orchestrators, pipelines, streaming integrations, and container runtimes that surround model development and deployment. section 3.4
- Vector databases that move similarity search from an in-memory Python experiment toward a persistent, scalable service. section 10.2.1
- Operational choices that trade off local simplicity against cloud scale, managed services, GPU acceleration, and data durability. section 10.2.1
A small but real deployment
For a local coding-copilot project, run Milvus Standalone with Docker, start it using the supplied shell script, load data, and stop or delete the container when finished. Small datasets can instead remain in memory with Faiss or USearch. section 10.2.1
Figure 3.7 A high level view of an MLOps infrastructure with LLMs in mind. This diagram attempts to cover the full picture and the complexity of the many tools involved to make ML models work in production.
Use the infrastructure diagram to inspect how LLM-oriented MLOps connects storage, pipelines, orchestration, containers, and model-serving components. section 3.4
