12 Holding on to your gains by watching for drift

In the preceding chapter, we established the foundations for measuring the effectiveness of an ML solution. This solid base enables a DS team to communicate to the business about the performance of a project in terms that are relevant to the business. To continue making (hopefully) positive reports about the effectiveness of a solution, a bit more work needs to be done.

If proper attribution monitoring and reporting to the business are the bedrock and foundation of a project, entropy is the buffeting storm seeking to continuously tear down the project. We call this chaotic shift in performance drift, and it takes many forms. Combatting against it requires continuous monitoring and a suspicious distrust of everything going into and coming out of a model.

Throughout this chapter, we will look at the types and causes of, and solutions for, the major types of model drift. Fighting against drift will help ensure that the gains that you’re making for your company continue to prove fruitful.

12.1 Detecting drift

12.1.1 What influences drift?

12.2 Responding to drift

12.2.1 What can we do about it?

12.2.2 Responding to drift

Summary