R-CNN

Also known as: Region-based convolutional neural network

On this page 7

R-CNN, short for region-based convolutional neural network, is a two-stage object detection method in computer vision that first proposes candidate image regions and then classifies each region with features from a convolutional neural network. Introduced in 2014, it was the first approach to show that deep CNN features learned on large classification datasets could dramatically improve detection accuracy on benchmarks such as PASCAL VOC.

The name now also covers a whole family of detectors: Fast R-CNN, Faster R-CNN and Mask R-CNN. Each successor kept the core idea of region-based reasoning while removing a bottleneck of the original, and the family remains a reference point for accuracy-oriented detection and instance segmentation.

How the original R-CNN pipeline works

The original R-CNN splits detection into four sequential stages, each trained or run separately. It does not look at the whole image at once; it reasons about roughly 2,000 candidate regions per image, which is the source of both its accuracy and its slowness.

StageWhat happensTechnique used
Region proposalAbout 2,000 candidate boxes are generatedSelective search, a non-learned grouping method
Region warpingEach box is cropped and resizedFixed input of 227 by 227 pixels
Feature extractionA CNN computes a feature vector per regionAlexNet-style network, 4,096 features
Classification and refinementRegions are scored and boxes adjustedLinear SVMs and bounding box regression
Stages of the original R-CNN

After scoring, non-maximum suppression removes overlapping boxes of the same class, keeping only the most confident detection. Regions are labeled during training using intersection over union with ground-truth boxes, typically treating an overlap of 0.5 or more as a positive for fine-tuning.

A key contribution was transfer learning: the CNN was pretrained on ImageNet classification and then fine-tuned on the smaller detection dataset. This showed that features learned for one task could transfer to localization, an idea that shaped most later work in the field.

How the R-CNN family evolved

Each successor in the R-CNN family removed one major inefficiency of its predecessor. The original ran the CNN separately on every region, so a single image required thousands of forward passes and inference took tens of seconds per image on a GPU of its time.

Fast R-CNN ran the CNN once over the full image and used RoI pooling to extract a fixed-size feature map for each proposal, training classification and box regression jointly. Faster R-CNN replaced selective search with a learned region proposal network that shares convolutional features with the detector and uses anchor boxes. Mask R-CNN added a parallel branch that predicts a pixel-level segmentation mask for each detected object and introduced RoIAlign to fix quantization errors.

ModelProposal methodMain improvementApproximate speed
R-CNNSelective searchCNN features for detectionTens of seconds per image
Fast R-CNNSelective searchShared features with RoI poolingAbout 2 seconds per image
Faster R-CNNRegion proposal networkEnd-to-end learned proposalsAbout 5 to 7 frames per second
Mask R-CNNRegion proposal networkInstance masks and RoIAlignAbout 5 frames per second
Evolution of the R-CNN family

Speeds depend heavily on backbone choice, input resolution and hardware, so the figures above indicate relative order rather than fixed values. Modern implementations with feature pyramid networks and current GPUs run considerably faster.

Two-stage detectors compared with single-stage detectors

R-CNN defined the two-stage detection paradigm, in which a model first asks where objects might be and then asks what they are. Single-stage detectors such as YOLO and SSD predict classes and boxes directly from a dense grid in one forward pass, trading some localization precision for speed.

PropertyTwo-stage (R-CNN family)Single-stage (YOLO, SSD)
ProcessingProposals, then per-region classificationDense prediction in one pass
Typical strengthHigher localization accuracy, small objectsReal-time speed, simpler deployment
Class imbalance handlingProposal stage filters backgroundNeeds focal loss or similar techniques
Common useBenchmarks, segmentation, offline analysisVideo, robotics, edge devices
Two-stage and single-stage detection

The accuracy gap has narrowed considerably. Modern single-stage detectors and transformer-based detectors often match or exceed Faster R-CNN on mean average precision, so the choice today depends more on ecosystem, latency budget and task than on a fixed accuracy ranking.

When to choose an R-CNN family detector

An R-CNN family model is a sound choice when precise boxes or masks matter more than latency. Faster R-CNN and Mask R-CNN remain widely used as baselines because their behavior is well understood and mature open-source implementations exist.

SituationRecommended choiceReason
Instance segmentation neededMask R-CNNProduces per-object masks with boxes
High-accuracy offline detectionFaster R-CNN with feature pyramidStrong localization on varied object sizes
Real-time video on edge hardwareSingle-stage detectorTwo-stage latency is usually too high
Teaching or historical comparisonOriginal R-CNNShows the pipeline stages explicitly
Decision guide for R-CNN family models

Limitations and common mistakes

The original R-CNN had well-known structural weaknesses that motivated its successors. These limitations are still useful to know because some of them persist in milder form across the family.

  • Multi-stage training: the original trained the CNN, the SVMs and the box regressors separately, which made tuning slow and fragile.
  • Heavy storage cost: features for every region were written to disk, often requiring hundreds of gigabytes for one dataset.
  • Warping distortion: resizing arbitrary boxes to a fixed square distorted object shapes and reduced accuracy for elongated objects.
  • Proposal dependence: an object missed by selective search can never be detected, whatever the classifier quality.
  • Confusing the family members: many references to R-CNN in practice mean Faster R-CNN, so model names should be stated precisely.

A non-technical view

R-CNN finds objects in a picture in two steps. First, it marks a few thousand spots that might contain something interesting. Then, it looks closely at each spot and decides what it is, such as a person, a car or a dog, and draws a box around it.

Think of a security guard checking a crowded room. Instead of studying every square inch, the guard first notes the places where people are standing and then looks carefully at each one. This approach is thorough and accurate but slow, and later versions learned to do the same work much faster.

A concrete example

A quality-control team at an electronics assembly plant inspects circuit boards for missing or misaligned components. Each high-resolution board image contains dozens of small parts, and the inspection runs offline after each production batch, so a few hundred milliseconds per image is acceptable.

The team trains a Faster R-CNN detector with a feature pyramid network on about 3,000 annotated board images, labeling capacitors, resistors and connectors. Because small components are critical, they tune anchor box sizes to the typical part dimensions and set the detection threshold at 0.6 confidence.

On a held-out test set, the model reaches a mean average precision of around 0.85 at an intersection over union of 0.5, and missed-component defects reach manual review far more consistently than with the previous rule-based system. The two-stage design suits the task because localization accuracy matters more than speed.