8 Experimentation in action: Finalizing an MVP with MLflow and runtime optimization

 

This chapter covers

  • Approaches, tools, and methods to version-control ML code, models, and experiment results
  • Scalable solutions for model training and inference

In the preceding chapter, we arrived at a solution to one of the most time-consuming and monotonous tasks that we face as ML practitioners: fine-tuning models. By having techniques to solve the tedious act of tuning, we can greatly reduce the risk of producing ML-backed solutions that are inaccurate to the point of being worthless. In the process of applying those techniques, however, we quietly welcomed an enormous elephant into the room of our project: tracking.

Throughout the last several chapters, we have been required to retrain our time-series models each time that we do inference. For the vast majority of other supervised learning tasks, this won’t be the case. Those other applications of modeling, both supervised and unsupervised, will have periodic retraining events, between which each model will be called for inference (prediction) many times.

8.1 Logging: Code, metrics, and results

8.1.1 MLflow tracking

8.1.2 Please stop printing and log your information

8.1.3 Version control, branch strategies, and working with others

8.2 Scalability and concurrency

8.2.1 What is concurrency?

8.2.2 What you can (and can’t) run asynchronously

Summary