concept good model in category machine learning

appears as: best model, good model, good models, A good model
Machine Learning Bookcamp

This is an excerpt from Manning's book Machine Learning Bookcamp.

A better approach for selecting the best model before deploying it is emulating the scenario of going live. We get our complete dataset and take a part out of it, keep the part away and train the model on the rest of the data. When the training is done, we pretend that the held-out dataset is the new, unseen data, and we use it to measure the performance of our models. This part of data is often called the validation set, and the process of keeping part of a dataset away and using it to evaluate performance is called validation (figure 1.10).

Figure 1.11 The validation process. We split the dataset into two parts, train models on the training part, and evaluate performance on the validation part. Using the evaluation results, we can choose the best model.

Often, we don't have two models to try, but a lot more. Logistic regression, for example, has a parameter, C, and depending on the value we set, the results can vary dramatically. Likewise, a neural network has many parameters, and each may have a great effect on the predictive performance of the final model. What's more, there are other models, each with its own set of parameters. How do we select the best model with the best parameters?

Real-World Machine Learning

This is an excerpt from Manning's book Real-World Machine Learning.

Although we haven’t programmed any of these functions yet, the basic structure is in place. By chapter 3, you’ll understand these steps; the rest of the book (chapters 4 through 10) is about making sure you’re building the best model for the problem at hand.

Figure 4.8. Comparison of the k-fold cross-validation error MSE to the MSE on new data, using the corn production dataset. The k-fold CV error is a good estimate for how the model will perform on new data, allowing you to use it confidently to forecast the error of the model and to select the best model.