Intersection over union (IoU) is a metric that measures the overlap between a predicted region and a ground truth region by dividing the area they share by the total area they cover together. In computer vision, it is the standard way to decide whether an object detector or a segmentation model has located an object correctly, and it is mathematically identical to the Jaccard index used for comparing sets.
IoU always lies between 0 and 1. A value of 0 means the regions do not touch, while a value of 1 means they coincide exactly. Because the score is normalized by the union, it is independent of object size, which allows a small pedestrian and a large truck to be judged on the same scale.
How intersection over union is calculated
IoU is computed as the area of intersection divided by the area of union, written as IoU = |A ∩ B| / |A ∪ B|. The union equals the sum of both areas minus their intersection, so the formula can also be expressed as |A ∩ B| / (|A| + |B| − |A ∩ B|).
For two axis-aligned bounding boxes, the intersection is found by taking the larger of the two left edges, the smaller of the two right edges and the corresponding top and bottom limits. If the resulting width or height is negative, the boxes do not overlap and IoU is 0. Implementations add a small epsilon or check for a zero union to avoid division errors on degenerate boxes.
The table below shows how simple geometric situations translate into IoU values for two boxes of equal size.
| Situation | Shared area | IoU |
|---|---|---|
| Identical boxes | 100% of each box | 1.00 |
| Shifted by one quarter of the width | 75% of each box | 0.60 |
| Shifted by half the width | 50% of each box | 0.33 |
| Shifted by three quarters of the width | 25% of each box | 0.14 |
| Touching edges only | 0% of each box | 0.00 |
The table reveals an important property: IoU falls faster than intuition suggests. Two boxes that share half of their area score only one third, which is why a threshold of 0.5 is stricter than it first appears.
IoU thresholds in detection benchmarks
In object detection, IoU serves as the matching rule that decides whether a prediction counts as a true positive. A predicted box is matched to a ground truth box of the same class when their IoU exceeds a chosen threshold; otherwise it counts as a false positive, and the unmatched ground truth counts as a false negative.
Benchmarks differ in how strict this rule is, and the choice changes reported accuracy considerably. The following table summarizes the most common conventions behind mean average precision scores.
| Convention | IoU threshold | What it rewards |
|---|---|---|
| PASCAL VOC metric | 0.5 | Finding the object roughly |
| COCO mAP | Average over 0.50 to 0.95 in steps of 0.05 | Precise localization across levels |
| AP75 in COCO | 0.75 | Tight boxes |
| 3D detection for vehicles | Often 0.7 | Accurate placement in space |
| Small or thin objects | Sometimes lowered below 0.5 | Tolerance for annotation noise |
Averaging across thresholds, as COCO does, prevents a model from scoring well merely by placing loose boxes near objects. It rewards consistently tight localization rather than detection alone.
IoU in segmentation and as a training loss
In image segmentation, IoU is computed pixel by pixel on masks instead of on rectangles. For semantic segmentation, the per-class scores are averaged into mean IoU (mIoU), the headline metric of most segmentation benchmarks. Instance and panoptic segmentation use mask IoU to match predicted instances to annotated ones.
IoU is also used directly as a loss function. Plain IoU loss has zero gradient when boxes do not overlap, so researchers introduced extensions that keep the signal informative. The table compares the main variants.
| Variant | Added term | Problem it solves |
|---|---|---|
| IoU loss | None, 1 − IoU | Scale-invariant box regression |
| GIoU | Penalty based on the smallest enclosing box | No gradient for disjoint boxes |
| DIoU | Normalized distance between box centers | Slow convergence of GIoU |
| CIoU | Distance plus aspect ratio consistency | Mismatched box shapes |
Generalized IoU ranges from −1 to 1 rather than 0 to 1, because the enclosing box penalty can push it below zero when boxes are far apart. This makes it usable as a loss but not directly comparable to plain IoU in evaluation reports.
Comparison with related overlap measures
IoU is closely related to the Dice coefficient, also known as the F1 score for sets. The two are monotonically linked by Dice = 2·IoU / (1 + IoU), so they rank predictions in the same order but produce different numbers; Dice is always equal to or higher than IoU.
| Measure | Formula idea | Typical use |
|---|---|---|
| Intersection over union | Shared area over combined area | Detection matching, segmentation benchmarks |
| Dice coefficient | Twice the shared area over the sum of areas | Medical image segmentation |
| Pixel accuracy | Correct pixels over all pixels | Quick segmentation sanity checks |
| Object keypoint similarity | Distance between keypoints scaled by object size | Pose estimation |
Pixel accuracy is misleading for unbalanced classes: a model that labels every pixel as background can score above 95% on images where the object is small, while its IoU for the object class is zero.
Limitations and common mistakes
IoU is simple and robust, but it has known blind spots that practitioners should keep in mind when interpreting scores or designing pipelines.
- Small objects are penalized heavily: an error of a few pixels can drop IoU sharply for a 10 by 10 pixel box.
- IoU ignores where the error lies: a box that cuts off the head of a person scores the same as one that adds background.
- Annotation noise caps achievable scores: two human annotators rarely agree above roughly 0.8 to 0.9 on the same object.
- Mixing coordinate formats corrupts results: confusing corner and center-width formats is a frequent implementation bug.
- Suppression and evaluation thresholds differ: thresholds in non-maximum suppression and evaluation are different settings and should not be tuned as if they were one parameter.
In plain words
Intersection over union is a way to grade how well a computer drew a box or outline around an object. Picture two sheets of tracing paper, one with the correct shape and one with the guess. Lay them on top of each other. The score is the overlapping part divided by everything covered by either sheet.
A perfect guess scores 1. A guess that misses completely scores 0. Anything above about 0.5 usually counts as finding the object, and higher scores mean the outline fits more tightly.
A real-world example
A logistics company trains a detector to locate parcels on a conveyor belt so that a robotic arm can pick them up. The first model reaches 0.91 average precision at an IoU threshold of 0.5, which looks excellent in the report.
On the line, however, the arm frequently grips the edge of a parcel and drops it. The engineers re-evaluate at a threshold of 0.75 and find that precision falls to 0.62, which shows that the boxes were roughly right but not tight enough for grasping.
They switch the box regression loss from smooth L1 to CIoU and clean up inconsistent annotations. Precision at the 0.75 threshold rises to 0.84, and failed picks drop by more than half, which confirms that the stricter IoU threshold matched the real requirement of the task.