liveBook · Manning

Cluster entropy

As explained in Timeless Algorithms, The Foundational Ideas

Cluster entropy measures the uncertainty in category membership among the observations within a cluster.

Cluster entropy measures the category uncertainty that remains within a cluster. Lower entropy indicates a more homogeneous group; unlike geometric measures, it reflects how informative cluster assignments are about known categories. section 5.3.4

Homogeneous fruit clusters

For 12 fruits grouped into three clusters of four—one cluster each of apples, bananas, and pears—knowing a fruit’s cluster leaves no uncertainty about its category. section 5.3.4

H= -[(1.0)      log2(1.0)]=0

The zero-entropy equation represents a cluster containing only one category. section 5.3.4

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines residual category uncertainty, contrasts it with geometry, and illustrates zero entropy using homogeneous fruit clusters.
Examples
Uses 12 fruits divided into pure apple, banana, and pear clusters to illustrate zero remaining uncertainty.
What it assumes
Assumes known categories for comparison, such as diagnoses, customer segments, or expert-defined labels.
What you can do after
Evaluate/understand whether clusters are homogeneous with respect to known categories.
How it connects
Connects cluster entropy with clustering evaluation, purity, normalized mutual information, and information entropy.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.