Non-maximum suppression (NMS) is a post-processing algorithm that removes redundant, overlapping predictions so that each detected object is represented by a single result. In computer vision, it is best known as the final step of object detection, where a detector typically proposes many bounding boxes around the same object and NMS keeps the highest-scoring box while discarding neighbors that overlap it too much.
The same idea appears earlier in image processing, for example in edge and corner detection, where NMS thins responses so that only local maxima survive. This page focuses on box-level NMS for detectors, the most common use today.
How non-maximum suppression selects boxes
Greedy NMS works on a list of candidate boxes, each with a confidence score and a class label. The algorithm first drops boxes below a score threshold, then sorts the remainder by score in descending order and processes them one by one.
- Select the box with the highest remaining score and add it to the output.
- Compute its intersection over union (IoU) with every remaining box of the same class.
- Remove each box whose IoU exceeds the IoU threshold, commonly between 0.45 and 0.7.
- Repeat until no candidates remain, then optionally cap the output at a maximum number of detections.
IoU is the area of overlap divided by the area of the union of two boxes, so it ranges from 0 for disjoint boxes to 1 for identical ones. Because each box is compared against the kept boxes, the naive implementation has quadratic complexity in the number of candidates, which is why detectors usually keep only the top few hundred or thousand boxes before running NMS.
The parameters below control most of the behavior and are worth tuning for each dataset.
| Parameter | Typical range | Effect of raising it |
|---|---|---|
| IoU threshold | 0.45 to 0.7 | Keeps more overlapping boxes, more duplicates |
| Score threshold | 0.001 to 0.5 | Fewer weak detections, lower recall |
| Maximum detections | 100 to 300 per image | More results, slower downstream steps |
| Pre-NMS top-k | 1,000 to 5,000 candidates | Better recall, higher compute cost |
Variants of non-maximum suppression
Greedy NMS makes a hard keep-or-delete decision, which is fast but loses information. Several variants change the suppression rule, the scope or the overlap measure to address specific weaknesses.
| Variant | Key idea | Best suited to |
|---|---|---|
| Greedy NMS | Delete boxes above the IoU threshold | General detection, fast inference |
| Soft-NMS | Decay scores of overlapping boxes instead of deleting | Crowded scenes with occlusion |
| Class-agnostic NMS | Suppress across all classes at once | Classes that are easily confused |
| DIoU-NMS | Add center distance to the overlap measure | Adjacent objects with similar boxes |
| Weighted box fusion | Average overlapping boxes by confidence | Ensembles and test-time augmentation |
| Matrix NMS | Parallel score decay in one matrix operation | Instance segmentation on GPUs |
Soft-NMS multiplies the scores of overlapping boxes by a linear or Gaussian penalty rather than removing them, which often improves mean average precision on crowded benchmarks at almost no extra cost. Batched NMS is an implementation trick rather than a new rule: it offsets box coordinates by class so that per-class suppression runs in a single GPU call.
The choice of variant should follow the data. Scenes with heavy occlusion, such as pedestrians or shelved products, benefit most from score decay, while ensemble pipelines gain more from fusing boxes than from deleting them. For real-time systems on edge hardware, plain greedy NMS with a tight candidate limit often remains the most practical option.
Non-maximum suppression in the detection pipeline
NMS sits between the raw detector output and the final predictions. Two-stage detectors such as R-CNN variants apply it twice: once to proposals from the region proposal network and once to the classified boxes. One-stage detectors such as YOLO apply it once, after decoding predictions from anchor boxes or anchor-free grids.
Some modern architectures are designed to avoid NMS entirely. The comparison below shows how the main detector families handle duplicate predictions.
| Detector family | Role of NMS | Consequence |
|---|---|---|
| Two-stage (R-CNN family) | Applied to proposals and final boxes | Accurate but more latency |
| One-stage (YOLO, SSD) | Single pass after prediction | Fast, sensitive to thresholds |
| DETR-style transformers | Not required | One-to-one matching learned in training |
| NMS-free one-stage designs | Removed by consistent label assignment | Lower and more predictable latency |
Transformer detectors achieve this through bipartite matching during training, which teaches each query to predict a distinct object. The trade-off is usually slower training convergence in exchange for a simpler and more deterministic inference step.
Limitations and tuning mistakes
The central weakness of NMS is that a single fixed threshold cannot fit every scene. A low threshold deletes genuine neighbors in a crowd, while a high threshold leaves duplicates on isolated objects. NMS is also not differentiable in its standard form, so the detector cannot learn to cooperate with it during training.
The table below lists typical symptoms seen in deployed detectors, with their usual cause and fix.
| Symptom | Likely cause | Typical fix |
|---|---|---|
| Missed people in crowds | IoU threshold too low | Raise threshold or use Soft-NMS |
| Duplicate boxes on one object | IoU threshold too high | Lower threshold to about 0.5 |
| Two labels on one object | Per-class NMS only | Add class-agnostic NMS step |
| Variable latency spikes | Too many candidates in busy frames | Limit pre-NMS top-k |
| Poor accuracy on rotated objects | Axis-aligned IoU used | Use rotated or polygon IoU |
A frequent evaluation mistake is to tune thresholds on the test set instead of a validation split, which inflates reported mean average precision. Another is to compare models that were evaluated with different NMS settings, since the threshold alone can shift scores noticeably.
Put simply
A detection model rarely draws just one box around a car. It often draws ten slightly different boxes around the same car. Non-maximum suppression is the cleanup step. It keeps the box the model is most sure about. Then it erases the other boxes that cover almost the same area.
Think of several people pointing at the same bird. NMS listens to the most confident pointer and ignores the rest. The main challenge is deciding how much overlap counts as the same object.
In practice
A retail analytics team runs a people-counting camera at a busy store entrance. Their YOLO-based detector produces about 40 candidate boxes per person before post-processing. With greedy NMS at an IoU threshold of 0.45, the system counted single shoppers correctly but undercounted groups walking shoulder to shoulder, because boxes of neighboring people overlapped by more than 45 percent and were deleted.
The engineers evaluated alternatives on a week of labeled footage. Raising the threshold to 0.6 created duplicates on lone shoppers, so they switched to Gaussian Soft-NMS and kept the score threshold at 0.3. The result was a count error below 3 percent during peak hours, with inference time per frame almost unchanged.