Intersection over union

Also known as: IoU · Jaccard index

On this page 7

Intersection over union (IoU) is a metric that measures the overlap between a predicted region and a ground truth region by dividing the area they share by the total area they cover together. In computer vision, it is the standard way to decide whether an object detector or a segmentation model has located an object correctly, and it is mathematically identical to the Jaccard index used for comparing sets.

IoU always lies between 0 and 1. A value of 0 means the regions do not touch, while a value of 1 means they coincide exactly. Because the score is normalized by the union, it is independent of object size, which allows a small pedestrian and a large truck to be judged on the same scale.

How intersection over union is calculated

IoU is computed as the area of intersection divided by the area of union, written as IoU = |A ∩ B| / |A ∪ B|. The union equals the sum of both areas minus their intersection, so the formula can also be expressed as |A ∩ B| / (|A| + |B| − |A ∩ B|).

For two axis-aligned bounding boxes, the intersection is found by taking the larger of the two left edges, the smaller of the two right edges and the corresponding top and bottom limits. If the resulting width or height is negative, the boxes do not overlap and IoU is 0. Implementations add a small epsilon or check for a zero union to avoid division errors on degenerate boxes.

The table below shows how simple geometric situations translate into IoU values for two boxes of equal size.

SituationShared areaIoU
Identical boxes100% of each box1.00
Shifted by one quarter of the width75% of each box0.60
Shifted by half the width50% of each box0.33
Shifted by three quarters of the width25% of each box0.14
Touching edges only0% of each box0.00
IoU values for two equal-size boxes

The table reveals an important property: IoU falls faster than intuition suggests. Two boxes that share half of their area score only one third, which is why a threshold of 0.5 is stricter than it first appears.

IoU thresholds in detection benchmarks

In object detection, IoU serves as the matching rule that decides whether a prediction counts as a true positive. A predicted box is matched to a ground truth box of the same class when their IoU exceeds a chosen threshold; otherwise it counts as a false positive, and the unmatched ground truth counts as a false negative.

Benchmarks differ in how strict this rule is, and the choice changes reported accuracy considerably. The following table summarizes the most common conventions behind mean average precision scores.

ConventionIoU thresholdWhat it rewards
PASCAL VOC metric0.5Finding the object roughly
COCO mAPAverage over 0.50 to 0.95 in steps of 0.05Precise localization across levels
AP75 in COCO0.75Tight boxes
3D detection for vehiclesOften 0.7Accurate placement in space
Small or thin objectsSometimes lowered below 0.5Tolerance for annotation noise
Common IoU thresholds and what they reward

Averaging across thresholds, as COCO does, prevents a model from scoring well merely by placing loose boxes near objects. It rewards consistently tight localization rather than detection alone.

IoU in segmentation and as a training loss

In image segmentation, IoU is computed pixel by pixel on masks instead of on rectangles. For semantic segmentation, the per-class scores are averaged into mean IoU (mIoU), the headline metric of most segmentation benchmarks. Instance and panoptic segmentation use mask IoU to match predicted instances to annotated ones.

IoU is also used directly as a loss function. Plain IoU loss has zero gradient when boxes do not overlap, so researchers introduced extensions that keep the signal informative. The table compares the main variants.

VariantAdded termProblem it solves
IoU lossNone, 1 − IoUScale-invariant box regression
GIoUPenalty based on the smallest enclosing boxNo gradient for disjoint boxes
DIoUNormalized distance between box centersSlow convergence of GIoU
CIoUDistance plus aspect ratio consistencyMismatched box shapes
IoU-based loss variants for bounding box regression

Generalized IoU ranges from −1 to 1 rather than 0 to 1, because the enclosing box penalty can push it below zero when boxes are far apart. This makes it usable as a loss but not directly comparable to plain IoU in evaluation reports.

IoU is closely related to the Dice coefficient, also known as the F1 score for sets. The two are monotonically linked by Dice = 2·IoU / (1 + IoU), so they rank predictions in the same order but produce different numbers; Dice is always equal to or higher than IoU.

MeasureFormula ideaTypical use
Intersection over unionShared area over combined areaDetection matching, segmentation benchmarks
Dice coefficientTwice the shared area over the sum of areasMedical image segmentation
Pixel accuracyCorrect pixels over all pixelsQuick segmentation sanity checks
Object keypoint similarityDistance between keypoints scaled by object sizePose estimation
Overlap measures used in computer vision

Pixel accuracy is misleading for unbalanced classes: a model that labels every pixel as background can score above 95% on images where the object is small, while its IoU for the object class is zero.

Limitations and common mistakes

IoU is simple and robust, but it has known blind spots that practitioners should keep in mind when interpreting scores or designing pipelines.

  • Small objects are penalized heavily: an error of a few pixels can drop IoU sharply for a 10 by 10 pixel box.
  • IoU ignores where the error lies: a box that cuts off the head of a person scores the same as one that adds background.
  • Annotation noise caps achievable scores: two human annotators rarely agree above roughly 0.8 to 0.9 on the same object.
  • Mixing coordinate formats corrupts results: confusing corner and center-width formats is a frequent implementation bug.
  • Suppression and evaluation thresholds differ: thresholds in non-maximum suppression and evaluation are different settings and should not be tuned as if they were one parameter.

In plain words

Intersection over union is a way to grade how well a computer drew a box or outline around an object. Picture two sheets of tracing paper, one with the correct shape and one with the guess. Lay them on top of each other. The score is the overlapping part divided by everything covered by either sheet.

A perfect guess scores 1. A guess that misses completely scores 0. Anything above about 0.5 usually counts as finding the object, and higher scores mean the outline fits more tightly.

A real-world example

A logistics company trains a detector to locate parcels on a conveyor belt so that a robotic arm can pick them up. The first model reaches 0.91 average precision at an IoU threshold of 0.5, which looks excellent in the report.

On the line, however, the arm frequently grips the edge of a parcel and drops it. The engineers re-evaluate at a threshold of 0.75 and find that precision falls to 0.62, which shows that the boxes were roughly right but not tight enough for grasping.

They switch the box regression loss from smooth L1 to CIoU and clean up inconsistent annotations. Precision at the 0.75 threshold rises to 0.84, and failed picks drop by more than half, which confirms that the stricter IoU threshold matched the real requirement of the task.