A recommender system predicts or ranks items, content, or actions for users or other entities. A production movie recommender combines structured interaction data, preparation pipelines, model training, validation, deployment, and monitoring rather than treating the recommendation algorithm as the whole system. section 1.4.2 section 9.3
A practical recommender workflow includes:
- Collect and prepare MovieLens ratings, then save reproducible outputs such as Parquet files and train, test, and validation splits. section 7.0 section 9.3
- Check incoming data quality, train the model, define validation metrics, and try the trained model to produce movie recommendations. section 9.3
- Track experiments and model versions with MLflow, while using Kubeflow and MinIO as supporting infrastructure for pipelines and stored data. section 9.3 section 7.0
- Monitor production behavior because new users, new items, changing preferences, and changing item popularity require retraining and drift detection. section 11.2.2
Detecting drift in a movie recommender
Train a matrix factorization model on a user-item rating matrix, increase selected movie ratings while keeping values within 1–5, retrain on drifted and unchanged data, and extract user and item factors. Wrap the factor distributions as tabular datasets and run deepchecks FeatureDrift; a changed item-factor distribution signals altered preferences or item popularity. section 11.2.2
Figure 7.1 The mental map where we’re now focusing on the first step/component of our pipeline—data collection and preparation (3)
Use the pipeline map to place data collection and preparation before training, validation, deployment, and operational monitoring. section 7.0
