MLflow is an open-source platform for managing machine-learning experimentation, reproducibility, packaging, deployment, and lifecycle operations. In LLM applications, it supports observability by centralizing tracking, model management, and deployment across providers and cloud platforms. section 11.4.7
Its production-oriented capabilities include:
- Packaging code, registering models, and deploying models through an MLflow server. section 11.4.7
- Supporting debugging and operational analysis alongside logs, KPIs, and request tracing. section 11.4.7
Track an LLM experiment
Configure an MLflow tracking URI, create a GenAI experiment, call a chat-completion API, and record latency, prompt tokens, conversation tokens, and completion tokens for each response.


