Suppose you need to group customers but do not know how many groups to expect. K-means cannot determine that number automatically: it takes the number of centroids as an argument. That choice is a hyperparameter—a value supplied to guide an algorithm, rather than a result the algorithm determines. Domain knowledge may suggest a value; otherwise, try several values and compare the resulting clusters. section 12.2.1
A hyperparameter controls different behavior depending on the algorithm:
- In DBSCAN, ε sets the radius of a dense region, while minPoints sets how many points are needed to define one. DBSCAN determines the number of clusters given those choices; unlike k-means, it does not require a cluster count. section 12.3.4
- In gradient descent, the learning rate α controls step size. Large steps can miss a minimum or diverge; small steps can make convergence too slow. A decreasing learning rate is one possible alternative to a fixed value. section 16.3.4
- In a force-directed graph drawing, β and δ balance attractive and repulsive forces. A balance that works for a sparse graph may pull a dense graph’s vertices toward the center. section 16.5.1
Choosing a cluster count
For customer segmentation, test several values of k. For each value, run k-means repeatedly, calculate intra-cluster distance, and retain the best result. Plot the results to use the elbow method as graphical feedback for choosing k. This is more useful than visual inspection when the dataset has too many dimensions to inspect directly. section 12.5 section 12.6
Figure 12.13 Using the first example dataset in figure 12.12, we can see how sensitive to the choice of parameters (in particular of ϵ) the result of DBSCAN is. All the examples ran DBSCAN on the same dataset with parameter minPoints set to 3 (the dimension of the domain plus 1). (Left) Using a value too small for ϵ causes the dense area to be too small, so data is partitioned into too many small clusters. A few points, plotted with X-shaped markers, are even marked as outliers. (Center) When the right value for the domain is chosen, the result is that DBSCAN partitions data perfectly into two concentric rings. (Right) When the radius ϵ is set to a value so big that the dense area of inner points extends to points of the outer ring, then the whole dataset is mistakenly assigned to the same cluster.
With minPoints fixed at 3, this comparison isolates the effect of ε: too small splits two rings into many clusters and outliers; too large merges both rings into one cluster. section 12.3.4
Tuning is constrained by the algorithm and the data. No cluster count fixes k-means’ difficulty with non-spherical clusters; DBSCAN can handle those shapes but may have no ε that partitions areas of different density correctly. Compare candidate results with evaluation metrics rather than assuming one setting works everywhere. section 12.2.1 section 12.3.4 section 12.5
