Evidently AI
Evidently AI is an open-source Python tool for evaluating, testing, monitoring, and reporting on machine-learning data, models, predictions, and behavior.
Machine Learning Platform Engineering
Evidently is an open-source tool for evaluating, testing, and monitoring machine-learning models and data.
- How it is used
- Monitors production feature distributions and supplies drift signals for dashboards, pipeline controls, and retraining.
- How deep it goes
- Explains drift categories, statistical tests, workspace reporting, and Kubeflow integration, but does not detail Evidently internals.
- Examples
- Uses an income classifier, covariate-shift distributions, drift tests, a Kubeflow component, and an Evidently dashboard.
- What you can do after
- Operate drift monitoring in inference pipelines, inspect reports, stop affected runs, and initiate timely retraining.
Read how Machine Learning Platform Engineering explains Evidently AI →
Machine Learning System Design
Evidently AI is an open-source Python library for evaluating, testing, and monitoring machine learning models and data.
- How it is used
- Uses an open-source Python library as an interim foundation for evaluating, testing, and monitoring machine learning systems.
- How deep it goes
- Lists monitoring domains and surrounding infrastructure, but defers foundational platform construction and detailed implementation.
- Examples
- Applies the tool to Supermegaretail demand forecasting with prediction logs, quality checks, operational metrics, and dashboards.
- What you can do after
- Operate/configure an initial ML monitoring setup and identify the logging and observability foundations it still requires.
Read how Machine Learning System Design explains Evidently AI →
Financial AI in Practice
Evidently is an open-source tool for evaluating and monitoring machine-learning data and models.
- How it is used
- Flags feature-distribution shifts and default-rate variation for credit-risk monitoring and compliance.
- How deep it goes
- Summarizes overall drift and provides per-feature and target-specific insights through interactive dashboards.
- Examples
- Visualizes feature-distribution drift and target drift reflected in default-rate changes.
- What you can do after
- Evaluate credit-model monitoring reports by reviewing overall drift, feature-level changes, and default-rate shifts.
Related concepts
- data driftuses · Data drift is a change over time in the statistical distribution or characteristics of data relative to a reference distribution or set of characteristics.
- monitoringuses · Monitoring is the ongoing observation and measurement of a system’s behavior, state, or operational signals to assess its condition and detect changes or problems.
- reference datasetuses · A reference dataset is a collection of observations designated as a basis for comparison, assessment, or analysis of other data.
- population stability indexmeasures · The population stability index (PSI) is a measure of the difference between two probability distributions, often used to quantify shifts in a population or model inputs.
- ScorecardMonitoringcontrasts with · ScorecardMonitoring is a scorecard-monitoring component that evaluates changes in score distributions and related statistical indicators.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.