liveBook · Manning

Entropy

As explained in Build AI Drug Discovery Pipelines

Entropy is an uncertainty measure based on the distribution of class probabilities.

Entropy measures uncertainty from the distribution of class probabilities. In decision-tree learning, it evaluates how well a feature split separates labeled examples. section 5.2.1

A decision tree uses entropy within a top-down, greedy construction process:

  • Start with labeled training examples and choose a feature and threshold that best separate their outcomes. section 5.2.1
  • Create branches, repeat the split process for each subset, and stop at conditions such as limited depth or homogeneous leaf labels. section 5.2.1
  • Compared with Gini impurity, entropy is more sensitive to changing class proportions and can sometimes produce more balanced trees under class imbalance. section 5.2.1

Molecular inhibition classification

A classifier can split molecules using RDKit descriptors such as molecular weight and logP, then predict cytochrome P450 inhibition for validation molecules. section 5.2.1

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains entropy’s role in split selection and class proportions, but does not provide its explicit mathematical formula.
Examples
Uses molecular weight, logP, cytochrome P450 inhibition labels, and validation molecules as decision-tree classification examples.
What it assumes
Assumes familiarity with features, class labels, training examples, and decision-tree branches.
What you can do after
Evaluate feature separations and interpret entropy’s role when training classification decision trees.
How it connects
Connects entropy with decision trees, Gini impurity, CART, class imbalance, feature thresholds, and leaf labels.

Also explained in

Compare how 19 books explain entropy →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.