Elbow method

On this page 7

The elbow method is a heuristic in machine learning for choosing the number of clusters k by plotting a clustering error measure against k and selecting the point where the curve bends, the elbow, after which additional clusters yield only diminishing returns. It is most often used with k-means clustering, where the error measure is the within-cluster sum of squares (WCSS), also called inertia.

The method is popular because it is simple, visual and cheap to compute for moderate data sizes. It is also widely criticized: real data often produce a smooth curve without a clear elbow, and the choice then becomes subjective. Practitioners therefore treat it as one input among several rather than as a definitive answer.

How the elbow method selects the number of clusters

The procedure fits the clustering algorithm repeatedly for a range of k values, typically from 1 to 10 or 15, and records a measure of within-cluster dispersion for each fit. For k-means, WCSS is the sum of squared Euclidean distances from each point to its assigned centroid.

  1. Scale the features so that no variable dominates the distance calculation.
  2. Run k-means for each candidate k, with several random initializations per k.
  3. Record the lowest WCSS obtained for each k.
  4. Plot WCSS on the vertical axis against k on the horizontal axis.
  5. Choose the k at which the slope changes most abruptly.

WCSS always decreases as k grows, reaching zero when every point is its own cluster, so the minimum of the curve is meaningless. The method looks for the point of maximum curvature, where the marginal gain from one more cluster drops from large to small. The table below shows a typical curve for data with four well-separated groups.

Number of clusters kWCSSReduction from previous k
110,000Not applicable
25,40046%
32,90046%
41,20059%
51,05013%
694010%
Example WCSS values for data with four true groups

Here the sharp drop ends at k = 4, after which each extra cluster removes only about a tenth of the remaining error, so four clusters is the reasonable choice.

Error measures and ways to locate the elbow

The elbow method works with any monotone dispersion measure, not only WCSS. The measure should match the algorithm: k-means minimizes squared distances, while a Gaussian mixture model optimizes likelihood, so plotting the wrong quantity produces a misleading curve.

MeasureTypical algorithmNote
WCSS or inertiak-meansDefault choice, sensitive to scale
Distortionk-meansMean distance to centroid, same shape as WCSS
Percentage of variance explainedAny centroid methodRatio of between-group to total variance
Negative log-likelihoodGaussian mixture modelBetter paired with BIC or AIC
Linkage distanceHierarchical clusteringRead from merge heights in a dendrogram
Measures used on the vertical axis

Visual inspection is subjective, so several automatic elbow detectors exist. The Kneedle algorithm normalizes the curve and finds the point farthest from the straight line joining its endpoints; the second-difference method picks the k with the largest change in slope. Both give reproducible choices, but they can disagree on smooth curves and depend on the chosen k range.

Comparison with other methods for choosing k

The elbow method is the simplest of several cluster validity approaches. Alternatives use different criteria, and their agreement or disagreement is itself informative about how strong the cluster structure really is.

MethodCriterionStrengthWeakness
Elbow methodBend in WCSS curveFast and intuitiveOften ambiguous
Silhouette analysisCohesion versus separation per pointClear maximum to selectCostly on large data
Gap statisticWCSS compared with random reference dataCan detect k = 1Needs many simulations
Calinski-Harabasz indexBetween- to within-cluster variance ratioCheap to computeFavors convex clusters
BIC or AICPenalized likelihoodPrincipled for mixture modelsRequires a probabilistic model
Methods for selecting the number of clusters

A notable weakness is that the elbow method cannot indicate that k = 1, meaning that the data have no cluster structure at all. The gap statistic addresses this by comparing the observed curve with curves from uniformly distributed reference data.

Practical guidance for applying the elbow method

Good results depend more on data preparation than on reading the plot. Feature scaling is essential, because k-means uses Euclidean distance and an unscaled variable measured in thousands will dominate one measured in fractions. High-dimensional data may also benefit from dimensionality reduction such as principal component analysis before clustering.

SituationRecommended actionReason
Clear elbow appearsAccept k, then confirm with silhouette scoresTwo criteria agree
Curve is smoothUse gap statistic or domain knowledgeNo objective bend exists
Two candidate elbowsInspect both solutions in detailStructure may be hierarchical
Very large datasetRun on a random sample or use mini-batch k-meansRepeated fits are expensive
Non-spherical clustersConsider DBSCAN or spectral clusteringWCSS assumes compact groups
Decisions when applying the method

Finally, the chosen k should be interpretable for the task. A statistically optimal five segments are of little value if the business can act on only three, so domain constraints legitimately influence the final choice.

Limitations and common mistakes

The method inherits every assumption of k-means: roughly spherical clusters of similar size and density. When those assumptions fail, WCSS can bend at a k that does not match any real structure.

  • Ambiguous curves are the norm on real data, so the selected elbow often reflects the analyst’s expectation.
  • Skipping feature scaling distorts distances and moves the elbow.
  • Single random initializations produce noisy curves with false bends.
  • Plot scaling matters: changing the axis range or aspect ratio can make an elbow appear or disappear.
  • Outliers inflate WCSS and may create a spurious elbow where a cluster absorbs them.

The simple explanation

Imagine sorting a pile of mixed coins into groups. Going from one group to two or three helps considerably, because coins become much more alike within each group. At some point, though, extra groups barely help, because the coins are already sorted well.

The elbow method draws a chart of how messy the groups still are for each number of groups. The line drops steeply and then flattens, like a bent arm. The bend is the best trade-off between too few groups and too many.

How it is used in practice

A data analyst at an online retailer wants to segment about 200,000 customers by purchase frequency, average order value and days since last purchase. After scaling the three features, the analyst runs k-means with 10 initializations for each k from 1 to 12 on a random sample of 20,000 customers and plots WCSS.

The curve falls steeply until k = 4 and then flattens, and the Kneedle algorithm also returns 4. Silhouette scores peak at 4 as well, so the analyst adopts four customer segments: frequent high spenders, frequent low spenders, occasional buyers and lapsed customers. The marketing team then designs a separate campaign for each segment, and the analyst reruns the method each quarter to detect shifts in customer behavior.