liveBook · Manning

Evaluation

As explained in Build AI Drug Discovery Pipelines

Model evaluation is the assessment and comparison of model predictions, including analysis of predictive errors.

Model evaluation assesses and compares predictions, analyzes errors, and guides iteration across a machine-learning workflow. In ligand-based screening, evaluation connects molecular features and labels with performance on data the model has not seen. section 3.0

A useful evaluation practice includes:

  • Separate training and test data before fitting the model, then judge generalization on the held-out test set rather than relying on training performance. section 3.3.2 section 3.5
  • Compare metrics derived from confusion matrices instead of treating a single score as sufficient. section 3.5
  • Use cross-validation to make efficient use of available training data, and revisit preprocessing, featurization, or training choices when performance is inadequate. section 3.0 section 3.5

Evaluating a hERG blocker classifier

A stochastic-gradient-descent classifier learns from molecular fingerprints after a train/test split. It predicts training labels and achieves 0.964 accuracy, but that result alone does not establish success: evaluation must address overfitting and performance on unseen molecules.

Figure 3.1 Key stages in our ML workflow.

Use the workflow diagram to trace how evaluation feeds decisions about data preparation, featurization, training, deployment, and maintenance. section 3.0

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Demonstrates train-set accuracy and discusses test data, confusion-matrix metrics, cross-validation, overfitting, and regularization without covering deployment maintenance.
Examples
Uses molecular fingerprints, a stochastic-gradient-descent classifier, hERG blocker labels, a train-test split, and a 0.964 training accuracy calculation.
What it assumes
Assumes molecular features, labels, training, and test sets; defines these workflow terms before applying them to classification.
What you can do after
Diagnose model quality with held-out data, classification metrics, cross-validation, and error analysis while recognizing overfitting.
How it connects
Connects model training, featurization, train-test splitting, confusion matrices, cross-validation, overfitting, regularization, deployment, and maintenance.

Also explained in

Compare how 29 books explain evaluation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.