Region proposal network

Also known as: RPN

On this page 7

A region proposal network (RPN) is a small, fully convolutional neural network that slides over a shared feature map and predicts, for a set of anchor boxes at each position, an objectness score and bounding box refinements, producing a short list of candidate object regions for a detector to classify. It was introduced with the Faster R-CNN detector and replaced slow hand-crafted proposal methods with a learned, nearly cost-free step.

Computer vision systems use the RPN as the first stage of two-stage object detection. Because it shares convolutional features with the second stage, the RPN made region-based detection fast enough for practical use in artificial intelligence applications and remains a core building block in models such as Mask R-CNN.

How a region proposal network works

An RPN works by placing a small sliding window, typically a 3×3 convolution, over the backbone feature map and mapping each position to an intermediate feature vector. Two sibling 1×1 convolution heads then read that vector: a classification head that outputs objectness scores and a regression head that outputs four box offsets per anchor.

At every feature map location the network evaluates k anchors, reference boxes of preset sizes and aspect ratios centered on that location. The original design used 3 scales and 3 aspect ratios, giving k = 9, so a feature map of 60 by 40 positions yields roughly 21,600 anchors per image. The table below lists the main stages of the pipeline.

StageOperationOutput
BackboneShared convolutional network extracts featuresFeature map, often at stride 16
Sliding window3×3 convolution over each positionIntermediate feature per location
Objectness head1×1 convolution with k or 2k outputsObject versus background scores
Regression head1×1 convolution with 4k outputsOffsets for center, width and height
Post-processingDecode boxes, clip, filter, suppress overlapsSeveral hundred to 2,000 proposals
Stages of the RPN pipeline

After decoding, proposals pass through non-maximum suppression with an intersection over union threshold of about 0.7, which removes near-duplicates. The surviving top-ranked boxes are handed to the second stage, which uses RoI pooling or RoIAlign to extract a fixed-size feature for each region.

Training targets and the loss function

An RPN is trained by labeling each anchor against the ground-truth boxes using intersection over union. Anchors with high overlap become positive examples, anchors with low overlap become negative examples, and ambiguous anchors are ignored during training so they do not send conflicting signals.

The loss combines a binary classification loss for objectness with a smooth L1 regression loss applied only to positive anchors. Because background anchors vastly outnumber object anchors, each training image samples a mini-batch of about 256 anchors with up to half positives. The usual labeling rules appear below.

Anchor conditionLabelEffect on training
IoU with any ground-truth box of at least 0.7PositiveTrains objectness and box regression
Highest IoU for a given ground-truth boxPositiveEnsures every object has an anchor
IoU below 0.3 with all ground-truth boxesNegativeTrains background classification only
IoU between 0.3 and 0.7IgnoredExcluded from the loss
Anchor crossing the image boundaryIgnored in trainingAvoids unstable regression targets
Typical anchor labeling rules

Box regression predicts normalized offsets rather than absolute coordinates: shifts of the center scaled by anchor width and height, and logarithmic ratios of width and height. This parameterization keeps targets in a stable numeric range across small and large objects.

Key parameters and design choices

The behavior of an RPN depends on a handful of configurable hyperparameters that trade recall against speed. Tuning them to the size and density of objects in the target domain often matters more than changing the backbone.

ParameterTypical valueWhat it affects
Anchor scales32 to 512 pixelsRange of object sizes that can be proposed
Aspect ratios0.5, 1 and 2Coverage of tall, square and wide objects
Pre-NMS top-N2,000 to 12,000 boxesCandidates considered before suppression
Post-NMS top-N300 to 2,000 boxesProposals passed to the second stage
NMS IoU thresholdAbout 0.7How aggressively overlapping proposals merge
Important RPN hyperparameters

Modern implementations attach an RPN to a feature pyramid network (FPN), assigning one anchor scale to each pyramid level. This multi-scale design greatly improves recall for small objects, which a single stride-16 feature map tends to miss.

How the RPN compares with other proposal methods

The RPN replaced hand-crafted proposal algorithms such as Selective Search, which merged superpixels on the CPU and needed about two seconds per image in the original R-CNN pipeline. By learning proposals from shared features, the RPN reduced that cost to roughly 10 milliseconds on a GPU.

MethodBasisSpeedTypical use
Selective SearchHierarchical grouping of superpixelsSlow, CPU-boundOriginal R-CNN and Fast R-CNN
EdgeBoxesContours enclosed by candidate boxesModerateEarly proposal benchmarks
Region proposal networkLearned anchors on shared featuresFast, GPU-friendlyFaster R-CNN and Mask R-CNN
One-stage dense headsDirect class and box prediction per anchorVery fastYOLO, SSD and RetinaNet
Learned object queriesSet prediction with attentionModerate to fastDETR-style transformers
Region proposal approaches compared

A one-stage detector such as YOLO can be seen as an RPN that predicts final classes directly instead of class-agnostic objectness. Two-stage designs with an RPN usually keep an edge in localization precision on crowded or small-object scenes, while one-stage detectors win on latency.

Limitations and common mistakes

An RPN inherits the weaknesses of anchor-based design. Practitioners most often run into the following issues:

  • Mismatched anchor sizes: objects far smaller or larger than any anchor receive no positive match and are never proposed.
  • Aggressive suppression: a strict NMS threshold merges proposals for adjacent objects in crowded scenes.
  • Too few proposals: cutting post-NMS top-N to speed up inference silently lowers detection recall.
  • Class imbalance: without balanced sampling, the objectness head learns to predict background everywhere.
  • Domain shift: anchors tuned on everyday photographs fit poorly on aerial, medical or document images.

The best diagnostic is to measure proposal recall at fixed IoU thresholds before tuning the second stage, because objects the RPN misses cannot be recovered later by the classifier.

In plain words

Finding objects in a photo is easier if someone first points at the spots worth a closer look. A region proposal network is that pointing assistant. It quickly scans the whole image and marks a few hundred boxes that probably contain something, whether a car, a person or a dog.

A second, more careful part of the system then examines only those boxes and decides what each one is. Because the first pass is fast and reuses the same image analysis, the whole detector becomes much quicker without losing accuracy.

A real-world example

A logistics company builds a system that counts parcels on a conveyor belt from overhead cameras. The engineers train a Faster R-CNN detector with an FPN backbone on 8,000 labeled images. At first the model misses small envelopes, so they inspect the region proposal network and find that the smallest anchor, 32 pixels, is larger than many envelopes at the camera’s resolution.

They add a 16-pixel anchor scale to the highest-resolution pyramid level, raise post-NMS proposals from 300 to 1,000 and retrain. Proposal recall at an IoU of 0.5 rises from about 88% to 97%, and final parcel counting errors fall noticeably, while inference still runs at around 12 frames per second on a single GPU, fast enough for the belt speed.