Model management is the ongoing administration of deployed models to sustain reliability, safety, and traceability. It is the final stage of LLMOps, linking serving, release control, monitoring, and governance. section 9.3.3
Core processes include:
- Deploy customized or pre-trained models through standardized APIs, using serving engines to manage compute and runtime performance. section 9.3.3
- Use gateways or proxies as a unified entry point across LLM providers. They can route requests, balance load, retry failures, and monitor costs. section 9.3.3
- Version the configuration that defines behavior: weights, fine-tuned variants, tokenizers, serving parameters, safety policies, and evaluation datasets. section 9.3.3
- Continuously monitor infrastructure, behavior, safety, and operational health using signals such as latency, throughput, token usage, faithfulness, and compliance. section 9.3.3
- Enforce policies and guardrails that keep interactions within safety and privacy boundaries. section 9.3.3
Figure 9.4 LLMOps lifecycle for context-aware systems, from pre-trained model to customization, caching, monitoring, versioning, and deployment
The lifecycle view places model management alongside customization, caching, monitoring, versioning, and deployment. section 9.3.3

