This chapter covers:
- Structuring testing environments, then migrating code and artifacts to them
- Measuring the properties of models
- Understanding how to test ML-discovered models offline and online
- Understanding how the test results can be used to select models
- Using qualitative evaluation and selection and quantitative measures
- Avoiding deceptive traps when evaluating your models
So far in sprint 2, the team designed the model to be developed using their understanding of the data, the client’s challenges and context, and the application that they expect to build. They’ve used a structured process to develop the model and tracked their progress using an experiment tracker and a model repository. They’ve also applied their common sense and experience to find and reject models that are suspicious or problematic. It’s important now to make sense of the model’s outcome and to properly evaluate competitor models to make good choices about the models that they take into production and application development.