Elbow method

As explained in Advanced Algorithms and Data Structures

The elbow method is a model-selection technique that chooses a parameter value at the point where additional complexity yields diminishing improvement in an evaluation measure.

The elbow method selects a hyper-parameter where additional model complexity produces sharply diminishing improvement. In clustering, plot an evaluation metric for several parameter values and choose the bend in the curve. section 12.5

A practical clustering workflow is:

• Choose an evaluation metric suited to the algorithm, such as intra-cluster average distance; centroid distance is specific to centroid-based methods, whereas intra-cluster distance also supports DBSCAN and OPTICS. section 12.5
• Test a sequence of parameter values and plot the best metric value for each one. For randomized k-means, run the algorithm several times per value and retain the best result. section 12.5
• Select the value at the curve’s elbow, where improvement changes substantially; after that point, added centroids improve the metric very little. section 12.5
• Inspect neighboring clusterings rather than relying on the plot alone: too few centroids can merge natural groups, while too many can split them. section 12.5.1

Choosing the number of customer clusters

For a scaled customer dataset, evaluate k-means across candidate values of k using intra-cluster average distance. The curve identifies k=6: k=5 leaves natural groups combined, while k=7 splits one natural group across two centroids. section 12.5 section 12.5.1

Figure 12.25 The elbow method used to determine the best value for the number of clusters into which the dataset in figure 12.24 should be split. You can guess where the name comes from.

Use the elbow plot to see how the selected parameter corresponds to the curve’s visible bend. section 12.5

How Advanced Algorithms and Data Structures covers it

How deep it goes

Explains metric choice, repeated randomized runs, curve interpretation, and neighboring-clustering checks without formalizing an automatic elbow threshold.

Examples

Uses a customer dataset, intra-cluster average distance, k values of 5 through 7, and pseudocode for repeated k-means runs.

What it assumes

Assumes familiarity with clustering algorithms, hyper-parameters, centroids, and evaluation metrics while distinguishing their algorithm-specific applicability.

What you can do after

Evaluate clustering hyper-parameters by running repeated trials, plotting metric results, selecting an elbow, and checking the resulting clusters.

How it connects

Connects clustering evaluation metrics with k-means, DBSCAN, OPTICS, and hyper-parameter selection.

liveBookmanning.com

Read Advanced Algorithms and Data Structures on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Statistics Slam DunkThe within-cluster sum of squares method selects a clustering count by examining how within-cluster squared distances decrease as the number of clusters increases.
• 

  liveBookmanning.com
  Data Without LabelsThe elbow method is a heuristic for selecting a model complexity value by locating a sharp bend in a curve of objective performance versus that value.
• 

  liveBookmanning.com
  The Well-Grounded Data AnalystThe elbow method selects an approximate number of clusters by identifying where additional clusters produce diminishing improvement in a clustering criterion.
• 

  liveBookmanning.com
  Cyber Threat HuntingThe elbow method is a clustering-validation technique that selects a cluster count by locating a point where additional clusters yield diminishing improvement in within-cluster distance.

Compare how 5 books explain elbow method →

Related concepts

• squared lossuses · Squared loss is the square of the difference between a predicted or fitted value and its target or observation, often summed across data points.
• k-means++ initializationuses · K-means++ initialization is a procedure that selects initial centroids using probability proportional to squared distance from previously selected centroids.
• K-means clusteringconfigures · K-means clustering partitions data into a specified number of groups by assigning observations to representative centers and iteratively updating those centers to reduce within-group distances.
• silhouette coefficientalternative to · The silhouette coefficient measures how well a sample fits its assigned cluster compared with the nearest alternative cluster, typically ranging from -1 to 1.
• hyperparameterconfigures · A hyperparameter is a setting chosen outside ordinary parameter learning that controls a model, algorithm, or learning procedure.
• within-cluster sum of squaresuses · Within-cluster sum of squares is the sum of squared distances between observations and the centroids of their assigned clusters.

This explainer was generated by Manning's knowledge graph from the text of Advanced Algorithms and Data Structures. The section links point to the parts of the book it draws on.