Object counting

Also known as: crowd counting

On this page 7

Object counting is the computer vision task of estimating the number of instances of one or more object categories in an image or video frame, such as people in a square, cells on a microscope slide or vehicles in a parking lot. Unlike tasks that must describe every object precisely, counting is judged mainly by how close the predicted total is to the true total.

When the objects are people in dense scenes, the task is usually called crowd counting, which has developed its own datasets and methods. Counting systems range from simple pipelines built on object detection to density map regression and newer class-agnostic models that count whatever category a user indicates with a few examples.

How object counting models estimate totals

Object counting methods fall into three broad families, each suited to a different object density and occlusion level. The choice of family usually matters more than the choice of a specific network.

ApproachHow the count is producedBest suited to
Counting by detectionDetect each object, then count the boxesSparse scenes with clearly separated objects
Density map regressionPredict a per-pixel density, then sum itDense crowds, cells and heavy occlusion
Direct regressionMap image features straight to one numberCoarse estimates with weak labels
Counting by segmentationSegment instances, then count the masksTouching objects with distinct shapes
Main approaches to object counting

In density map regression, each annotated object is represented by a point, and training targets are created by placing a small Gaussian kernel at each point. Because every kernel integrates to one, summing the predicted density map returns the estimated count, even when individual objects are only a few pixels wide and cannot be detected reliably.

Annotation and training data for counting

Counting datasets rely on point annotations far more often than bounding boxes, because marking a single point per object is fast enough for images that contain thousands of instances. A typical dense crowd image may require one to several thousand clicks, which is still cheaper than drawing boxes or masks.

The width of the Gaussian kernel is a key design parameter. A fixed kernel suits objects of similar size, while geometry-adaptive kernels scale with the distance to neighboring points to approximate perspective, so distant heads receive narrower kernels than nearby ones.

Annotation typeEffort per objectSupports
Image-level countVery low, one number per imageDirect regression only
Point annotationLow, one clickDensity maps and point-based models
Bounding boxMedium, two cornersCounting by detection
Instance maskHigh, full outlineCounting by segmentation
Annotation types used in object counting

Metrics that measure counting accuracy

Counting accuracy is reported with error metrics on the total per image, not with the detection metrics used for localization. A model can score well on counting while placing some predictions in the wrong location, so localization quality, when it matters, must be measured separately.

MetricDefinitionWhat it reveals
Mean absolute errorAverage of the absolute count differencesTypical size of the error
Root mean squared errorSquare root of the mean squared differenceSensitivity to large failures
Relative errorAbsolute error divided by the true countError comparable across densities
Grid average mean absolute errorError computed over image sub-regionsWhether counts are placed correctly
Common evaluation metrics for counting

A large gap between root mean squared error and mean absolute error indicates that a few images, often the densest or most unusual ones, produce very large mistakes. Reporting both values is standard practice on crowd counting benchmarks.

Object counting compared with neighboring tasks

Object counting overlaps with several computer vision tasks but asks a narrower question. The comparison below shows what each task outputs and why counting is often solved without full localization.

TaskMain outputRelationship to counting
Object detectionBoxes with class labelsCount equals number of kept boxes
Instance segmentationOne mask per objectCount equals number of masks
Object trackingIdentities across framesNeeded to count unique objects in video
Object countingA number or density mapOptimized directly for the total
Counting and related vision tasks

Video adds a distinct problem: counting unique objects over time. Summing per-frame counts overcounts any object that stays in view, so line-crossing counters for traffic or pedestrians combine detection with object tracking and count an object only when its track crosses a virtual line.

Limitations and common sources of error

Counting errors usually stem from scene conditions that differ from the training data. Practitioners should expect accuracy to drop when deploying a model in a new camera position, lighting condition or object density.

  • Scale variation: the same object may span hundreds of pixels near the camera and only a few pixels far away.
  • Occlusion: overlapping people, fruit or cells hide parts of objects, which defeats detection-based counting.
  • Background clutter: textures such as foliage or patterned floors can be mistaken for dense objects.
  • Duplicate suppression errors: aggressive non-maximum suppression merges touching objects and undercounts them.
  • Domain shift: a model trained on daytime street crowds may fail on stadium or night scenes.

A practical safeguard is to validate on images from the target site with manual counts before relying on automated totals. Where counts carry legal or safety weight, such as occupancy limits, systems should report an uncertainty range rather than a single exact figure.

The short version

Object counting means teaching a computer to answer one question: how many are there. It does not need to name every item in a picture. It focuses on the total, such as people in a square, apples on a tree or cars in a parking lot.

For crowded scenes, the computer does not outline each person. It estimates how densely packed each part of the image is. Then it adds those estimates together. A person might do the same by counting one small patch of a crowd and multiplying across the whole area.

Where it makes a difference

An agricultural cooperative wants to forecast its apple harvest several weeks before picking. A field team photographs sample rows of trees from both sides with a tractor-mounted camera, producing about 4,000 images per orchard block.

A density-based counting model, fine-tuned on 300 images with point annotations of visible fruit, estimates apples per tree. Because leaves hide part of the fruit, the cooperative calibrates the model against a manual count of 40 trees and applies a correction factor for occlusion.

The resulting forecast falls within roughly 5 to 8 percent of the final harvested volume, compared with the much wider spread of earlier visual estimates. Accurate counts let the cooperative book storage capacity and seasonal pickers earlier and with less waste.