The elbow method is a heuristic in machine learning for choosing the number of clusters k by plotting a clustering error measure against k and selecting the point where the curve bends, the elbow, after which additional clusters yield only diminishing returns. It is most often used with k-means clustering, where the error measure is the within-cluster sum of squares (WCSS), also called inertia.
The method is popular because it is simple, visual and cheap to compute for moderate data sizes. It is also widely criticized: real data often produce a smooth curve without a clear elbow, and the choice then becomes subjective. Practitioners therefore treat it as one input among several rather than as a definitive answer.
How the elbow method selects the number of clusters
The procedure fits the clustering algorithm repeatedly for a range of k values, typically from 1 to 10 or 15, and records a measure of within-cluster dispersion for each fit. For k-means, WCSS is the sum of squared Euclidean distances from each point to its assigned centroid.
- Scale the features so that no variable dominates the distance calculation.
- Run k-means for each candidate k, with several random initializations per k.
- Record the lowest WCSS obtained for each k.
- Plot WCSS on the vertical axis against k on the horizontal axis.
- Choose the k at which the slope changes most abruptly.
WCSS always decreases as k grows, reaching zero when every point is its own cluster, so the minimum of the curve is meaningless. The method looks for the point of maximum curvature, where the marginal gain from one more cluster drops from large to small. The table below shows a typical curve for data with four well-separated groups.
| Number of clusters k | WCSS | Reduction from previous k |
|---|---|---|
| 1 | 10,000 | Not applicable |
| 2 | 5,400 | 46% |
| 3 | 2,900 | 46% |
| 4 | 1,200 | 59% |
| 5 | 1,050 | 13% |
| 6 | 940 | 10% |
Here the sharp drop ends at k = 4, after which each extra cluster removes only about a tenth of the remaining error, so four clusters is the reasonable choice.
Error measures and ways to locate the elbow
The elbow method works with any monotone dispersion measure, not only WCSS. The measure should match the algorithm: k-means minimizes squared distances, while a Gaussian mixture model optimizes likelihood, so plotting the wrong quantity produces a misleading curve.
| Measure | Typical algorithm | Note |
|---|---|---|
| WCSS or inertia | k-means | Default choice, sensitive to scale |
| Distortion | k-means | Mean distance to centroid, same shape as WCSS |
| Percentage of variance explained | Any centroid method | Ratio of between-group to total variance |
| Negative log-likelihood | Gaussian mixture model | Better paired with BIC or AIC |
| Linkage distance | Hierarchical clustering | Read from merge heights in a dendrogram |
Visual inspection is subjective, so several automatic elbow detectors exist. The Kneedle algorithm normalizes the curve and finds the point farthest from the straight line joining its endpoints; the second-difference method picks the k with the largest change in slope. Both give reproducible choices, but they can disagree on smooth curves and depend on the chosen k range.
Comparison with other methods for choosing k
The elbow method is the simplest of several cluster validity approaches. Alternatives use different criteria, and their agreement or disagreement is itself informative about how strong the cluster structure really is.
| Method | Criterion | Strength | Weakness |
|---|---|---|---|
| Elbow method | Bend in WCSS curve | Fast and intuitive | Often ambiguous |
| Silhouette analysis | Cohesion versus separation per point | Clear maximum to select | Costly on large data |
| Gap statistic | WCSS compared with random reference data | Can detect k = 1 | Needs many simulations |
| Calinski-Harabasz index | Between- to within-cluster variance ratio | Cheap to compute | Favors convex clusters |
| BIC or AIC | Penalized likelihood | Principled for mixture models | Requires a probabilistic model |
A notable weakness is that the elbow method cannot indicate that k = 1, meaning that the data have no cluster structure at all. The gap statistic addresses this by comparing the observed curve with curves from uniformly distributed reference data.
Practical guidance for applying the elbow method
Good results depend more on data preparation than on reading the plot. Feature scaling is essential, because k-means uses Euclidean distance and an unscaled variable measured in thousands will dominate one measured in fractions. High-dimensional data may also benefit from dimensionality reduction such as principal component analysis before clustering.
| Situation | Recommended action | Reason |
|---|---|---|
| Clear elbow appears | Accept k, then confirm with silhouette scores | Two criteria agree |
| Curve is smooth | Use gap statistic or domain knowledge | No objective bend exists |
| Two candidate elbows | Inspect both solutions in detail | Structure may be hierarchical |
| Very large dataset | Run on a random sample or use mini-batch k-means | Repeated fits are expensive |
| Non-spherical clusters | Consider DBSCAN or spectral clustering | WCSS assumes compact groups |
Finally, the chosen k should be interpretable for the task. A statistically optimal five segments are of little value if the business can act on only three, so domain constraints legitimately influence the final choice.
Limitations and common mistakes
The method inherits every assumption of k-means: roughly spherical clusters of similar size and density. When those assumptions fail, WCSS can bend at a k that does not match any real structure.
- Ambiguous curves are the norm on real data, so the selected elbow often reflects the analyst’s expectation.
- Skipping feature scaling distorts distances and moves the elbow.
- Single random initializations produce noisy curves with false bends.
- Plot scaling matters: changing the axis range or aspect ratio can make an elbow appear or disappear.
- Outliers inflate WCSS and may create a spurious elbow where a cluster absorbs them.
The simple explanation
Imagine sorting a pile of mixed coins into groups. Going from one group to two or three helps considerably, because coins become much more alike within each group. At some point, though, extra groups barely help, because the coins are already sorted well.
The elbow method draws a chart of how messy the groups still are for each number of groups. The line drops steeply and then flattens, like a bent arm. The bend is the best trade-off between too few groups and too many.
How it is used in practice
A data analyst at an online retailer wants to segment about 200,000 customers by purchase frequency, average order value and days since last purchase. After scaling the three features, the analyst runs k-means with 10 initializations for each k from 1 to 12 on a random sample of 20,000 customers and plots WCSS.
The curve falls steeply until k = 4 and then flattens, and the Kneedle algorithm also returns 4. Silhouette scores peak at 4 as well, so the analyst adopts four customer segments: frequent high spenders, frequent low spenders, occasional buyers and lapsed customers. The marketing team then designs a separate campaign for each segment, and the analyst reruns the method each quarter to detect shifts in customer behavior.