Reference dataset
A reference dataset is a collection of observations designated as a basis for comparison, assessment, or analysis of other data.
Outlier Detection in Python
A reference dataset is a collection of observations used as a basis for comparing other observations or assessing what is typical.
- How it is used
- Supports outlier detection by defining the observations against which new or local observations are compared.
- How deep it goes
- Explains selection trade-offs, temporal representativeness, quality checks, neighborhood construction, and prediction cost without resolving a universal choice.
- Examples
- Uses click histories, a defect-free industrial period, rolling monitoring data, and a 35-feature synthetic SOD dataset.
- What you can do after
- Evaluate and configure reference data for anomaly comparisons while balancing representativeness, quality, detection reliability, and prediction cost.
Read how Outlier Detection in Python explains reference dataset →
Machine Learning Platform Engineering
A reference dataset is a feature dataset containing the features associated with users across the training and testing populations.
- How it is used
- Anchors model validation, data-quality monitoring, and production drift detection against historical feature data.
- How deep it goes
- Explains dataset composition, storage, retrieval, comparison, reporting, and pipeline gating without detailing statistical-test internals.
- Examples
- Uses training and test features, MLflow-logged files, Feast inference features, and Evidently drift dashboards as concrete artifacts.
- What you can do after
- Operate reproducible drift monitoring by retrieving a model-linked baseline and comparing it with current inference features.
Read how Machine Learning Platform Engineering explains reference dataset →
Related concepts
- Evidently AIused by · Evidently AI is an open-source Python tool for evaluating, testing, monitoring, and reporting on machine-learning data, models, predictions, and behavior.
- shared neighbors distanceproduced by · Shared neighbors distance measures dissimilarity between observations according to the neighbors they have in common.
- local outlier factorused by · Local Outlier Factor is a density-based anomaly-detection algorithm that scores an observation by comparing its local density with those of its neighbors.
- k-nearest neighborsused by · K-nearest neighbors is a method that predicts a query’s class or value from the labels or values of its k closest training examples, as measured by a specified distance or similarity.
- data driftused by · Data drift is a change over time in the statistical distribution or characteristics of data relative to a reference distribution or set of characteristics.
- subspace outlier detectionused by · Subspace outlier detection is an outlier detection method that identifies anomalies within feature subspaces selected from local reference sets.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.