Person re-identification

Also known as: ReID

On this page 7

Person re-identification (ReID) is the computer vision task of recognizing the same individual across images or video captured by different, usually non-overlapping cameras, at different times and from different viewpoints. Given a query image of a person, a ReID system ranks a gallery of previously captured images by how likely each one shows the same identity.

Unlike facial recognition, person re-identification typically relies on whole-body appearance, such as clothing, body shape, carried objects and gait, because faces are often too small, turned away or occluded in surveillance footage. The task is therefore framed as retrieval by learned similarity rather than as classification into a fixed list of known people.

How person re-identification matches identities

A modern ReID pipeline converts each person crop into a fixed-length embedding vector and compares embeddings with a distance metric. Crops come from an upstream object detection stage that produces a bounding box for every pedestrian, and often from object tracking, which groups detections into short tracklets within one camera.

The network, usually a ResNet-50 backbone or a vision transformer, is trained so that images of the same person lie close together in embedding space and images of different people lie far apart. At query time, the system computes the cosine or Euclidean distance between the query embedding and every gallery embedding, then sorts the gallery by that distance.

The table below outlines the main stages and what each contributes.

StageInputOutput
DetectionRaw video framesPerson bounding boxes
TrackingBoxes over time in one cameraTracklets of one person
Feature extractionCropped person imagesEmbedding vectors, often 512 to 2048 values
Matching and rankingQuery and gallery embeddingsRanked list of candidate identities
Re-rankingInitial ranked listRefined list using neighborhood structure
Stages of a typical ReID pipeline

Training commonly combines an identity classification loss with a triplet loss, which pulls an anchor toward a positive example and pushes it away from a negative one by a margin. Techniques such as batch-hard mining, part-based features and k-reciprocal re-ranking improve accuracy on difficult cases.

Variants and settings of the ReID task

Person re-identification comes in several variants that differ in input modality and supervision. The choice of variant determines which datasets, models and accuracy expectations apply.

The following table compares the most common settings.

VariantWhat is matchedTypical challenge
Image-based ReIDSingle crops against single cropsPose and viewpoint changes
Video-based ReIDTracklets against trackletsAggregating frames of varying quality
Visible-infrared ReIDDaytime color images and night infrared imagesLarge modality gap
Occluded ReIDPartially hidden personsMissing body parts
Clothes-changing ReIDSame person in different outfitsClothing cues become misleading
Unsupervised ReIDUnlabeled target-domain dataNo identity labels for training
Common ReID settings

Most published systems assume short-term ReID, where people keep the same clothing between sightings. Long-term and clothes-changing settings are far harder because the strongest visual cue disappears, forcing models toward body shape, gait and head features.

Evaluation metrics and benchmarks

ReID performance is measured with retrieval metrics, chiefly the cumulative matching characteristic (CMC) at rank-k and mean average precision. Rank-1 accuracy reports how often the top-ranked gallery image is a correct match, while mAP rewards placing all correct matches high in the list.

The table summarizes the metrics and what each reveals.

MetricWhat it measuresInterpretation
Rank-1 accuracyTop result is the correct identityUsability for quick lookups
Rank-5 and rank-10Correct identity within the first k resultsUsefulness for human review lists
Mean average precisionRanking quality over all true matchesOverall retrieval quality
mINPCost of finding the hardest true matchEffort needed to retrieve every sighting
Core evaluation metrics

Widely used academic benchmarks include Market-1501, MSMT17 and the now-retired DukeMTMC-reID. Leading models report rank-1 accuracy above 95 percent on Market-1501 but considerably lower scores on MSMT17, and accuracy drops sharply in cross-domain evaluation, where a model trained in one camera network is tested in another.

Person re-identification overlaps with several neighboring computer vision tasks but differs in its open-set, cross-camera retrieval formulation. The distinctions clarify where ReID fits in a larger system.

  • Facial recognition matches faces and needs a visible, reasonably large face; ReID works on full-body appearance at low resolution.
  • Object tracking follows a target within one continuous view; ReID links identities across camera gaps and time gaps.
  • Image classification assigns one of a fixed set of labels; ReID handles identities never seen during training.
  • Content-based image retrieval is the general family to which ReID belongs, specialized for pedestrians.

In multi-camera tracking systems, ReID features are frequently combined with motion and timing constraints, because appearance alone is ambiguous when many people wear similar clothing.

Limitations, privacy and failure modes

Person re-identification fails most often under appearance ambiguity and domain shift, and it raises serious privacy and civil-liberty concerns because it enables following individuals across public spaces without their knowledge.

The table pairs typical symptoms with their cause and a common remedy.

SymptomLikely causeTypical remedy
Wrong match in similar uniformsClothing dominates the embeddingPart-based or attribute-aware features
Accuracy collapse at a new siteDomain shift in lighting and camerasDomain adaptation or target-site fine-tuning
Misses for partly hidden peopleOcclusion removes key regionsVisibility-aware part matching
Poor results at nightInfrared and color images differCross-modality training
Uneven accuracy across groupsUnbalanced training dataDataset auditing and balanced sampling
Common failure modes and remedies

In the European Union, appearance-based tracking of identifiable people counts as processing of personal data under the GDPR, and the AI Act places strict limits on remote biometric identification in public spaces. Responsible deployments therefore apply purpose limitation, short retention and human review of every match.

Understanding it simply

Person re-identification answers a simple question: is the person in this camera the same person seen in another camera? It does this without reading a name or seeing a face clearly, by comparing overall appearance such as clothing, build and the bag someone carries.

It works a little like spotting a friend in a crowded station by their red coat and backpack. The system turns each picture into a numeric fingerprint and looks for the closest fingerprints. It can be very accurate, but it is easily fooled when people dress alike or change clothes.

Applied in practice

A large international airport operates about 400 cameras in its terminals. When a parent reports a lost seven-year-old near Gate B12, security staff upload a photo taken that morning at check-in. The ReID system compares it with embeddings from the last 30 minutes of footage across all terminal cameras.

Within seconds, the system returns a ranked list of 20 candidate sightings. An operator reviews the top results, confirms that ranks 1, 2 and 4 show the child walking toward a food court, and dispatches staff to the right location in under five minutes. Rank 3 is a different child in a similar jacket, which the operator discards.

Because the search is limited to an active incident, the airport keeps embeddings for only 72 hours and logs every query. Human confirmation of each match prevents a false identification from triggering action against the wrong family.