liveBook · Manning

Clustering

As explained in The Art of AI Product Development

Clustering is an unsupervised learning technique that groups data points according to similarities in their characteristics.

A product team may know its users’ demographics but still struggle to explain why some embrace recommendations while others only browse. Clustering groups similar data points without labeled learning targets, helping teams discover patterns in data that is not yet well understood. The resulting segments can inform personalization, marketing, and feature decisions, but the patterns found are not automatically useful. section 4.1 section 3.1.2 section 4.1.5

For behavioral segmentation, choose features that reflect the product question. Recommendation clicks, purchases, browsing, and profile edits can reveal patterns that demographic categories miss. Clean and aggregate the data, then use domain expertise to select, scale, and refine features: a valid grouping based on irrelevant features may be of little practical value. section 4.1.1 section 4.1.3 section 4.1.5

Different methods expose different structures, so compare them against the data and the insight sought:

  • K-means assigns points to their nearest center, recalculates centers from cluster averages, and repeats until groups stabilize. It requires a preset number of clusters. section 4.1.3
  • Hierarchical clustering builds a tree of clusters, allowing exploration from broader groups to more specific behaviors without predefining the number of clusters. section 4.1.3
  • DBSCAN identifies dense groups and can reveal arbitrary-shaped clusters and outliers, unlike the spherical clusters assumed by K-means. section 4.1.3
Figure 4.2 K-means clustering looks for centroids surrounded by an agglomeration of data points.
A diagram of different colored dots

AI-generated content may be incorrect.

The figure illustrates K-means centers surrounded by groups of data points. section 4.1.3

Evaluate whether segments are distinct and useful. The silhouette coefficient reflects similarity within clusters relative to other clusters; the Calinski-Harabasz index helps assess separation and choose a cluster count. Iterate on features, preprocessing, and algorithm parameters. Too few groups can conceal meaningful differences; too many can overwhelm the team expected to act on them. section 4.1.4 section 4.1.5

From segments to action

An e-commerce team identifies users who frequently edit profiles, ignore recommendations, browse without buying, or engage deeply and purchase often. Visualizing and describing the groups helps the team tailor experiences. An anomalous user who clicks endlessly without buying turns out to be a scraping bot, prompting a security alert. To assign new users without repeatedly rerunning clustering, the team proposes training a supervised classifier on the existing segments. section 4.1.6

How The Art of AI Product Development covers it

How deep it goes
Explains three clustering methods, evaluation metrics, and iterative feature and cluster-count optimization without implementation code or mathematical derivations.
Examples
Uses e-commerce recommendation engagement, purchases, profile edits, and a scraping bot to illustrate behavioral segments and anomalies.
What you can do after
Evaluate and understand clustering options, then use behavioral segments to guide product and marketing actions.
How it connects
Connects unsupervised learning and feature engineering with classification, anomaly detection, and personalization.

Also explained in

Compare how 19 books explain clustering →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of The Art of AI Product Development. The section links point to the parts of the book it draws on.