Person re-identification (ReID) is the computer vision task of recognizing the same individual across images or video captured by different, usually non-overlapping cameras, at different times and from different viewpoints. Given a query image of a person, a ReID system ranks a gallery of previously captured images by how likely each one shows the same identity.
Unlike facial recognition, person re-identification typically relies on whole-body appearance, such as clothing, body shape, carried objects and gait, because faces are often too small, turned away or occluded in surveillance footage. The task is therefore framed as retrieval by learned similarity rather than as classification into a fixed list of known people.
How person re-identification matches identities
A modern ReID pipeline converts each person crop into a fixed-length embedding vector and compares embeddings with a distance metric. Crops come from an upstream object detection stage that produces a bounding box for every pedestrian, and often from object tracking, which groups detections into short tracklets within one camera.
The network, usually a ResNet-50 backbone or a vision transformer, is trained so that images of the same person lie close together in embedding space and images of different people lie far apart. At query time, the system computes the cosine or Euclidean distance between the query embedding and every gallery embedding, then sorts the gallery by that distance.
The table below outlines the main stages and what each contributes.
| Stage | Input | Output |
|---|---|---|
| Detection | Raw video frames | Person bounding boxes |
| Tracking | Boxes over time in one camera | Tracklets of one person |
| Feature extraction | Cropped person images | Embedding vectors, often 512 to 2048 values |
| Matching and ranking | Query and gallery embeddings | Ranked list of candidate identities |
| Re-ranking | Initial ranked list | Refined list using neighborhood structure |
Training commonly combines an identity classification loss with a triplet loss, which pulls an anchor toward a positive example and pushes it away from a negative one by a margin. Techniques such as batch-hard mining, part-based features and k-reciprocal re-ranking improve accuracy on difficult cases.
Variants and settings of the ReID task
Person re-identification comes in several variants that differ in input modality and supervision. The choice of variant determines which datasets, models and accuracy expectations apply.
The following table compares the most common settings.
| Variant | What is matched | Typical challenge |
|---|---|---|
| Image-based ReID | Single crops against single crops | Pose and viewpoint changes |
| Video-based ReID | Tracklets against tracklets | Aggregating frames of varying quality |
| Visible-infrared ReID | Daytime color images and night infrared images | Large modality gap |
| Occluded ReID | Partially hidden persons | Missing body parts |
| Clothes-changing ReID | Same person in different outfits | Clothing cues become misleading |
| Unsupervised ReID | Unlabeled target-domain data | No identity labels for training |
Most published systems assume short-term ReID, where people keep the same clothing between sightings. Long-term and clothes-changing settings are far harder because the strongest visual cue disappears, forcing models toward body shape, gait and head features.
Evaluation metrics and benchmarks
ReID performance is measured with retrieval metrics, chiefly the cumulative matching characteristic (CMC) at rank-k and mean average precision. Rank-1 accuracy reports how often the top-ranked gallery image is a correct match, while mAP rewards placing all correct matches high in the list.
The table summarizes the metrics and what each reveals.
| Metric | What it measures | Interpretation |
|---|---|---|
| Rank-1 accuracy | Top result is the correct identity | Usability for quick lookups |
| Rank-5 and rank-10 | Correct identity within the first k results | Usefulness for human review lists |
| Mean average precision | Ranking quality over all true matches | Overall retrieval quality |
| mINP | Cost of finding the hardest true match | Effort needed to retrieve every sighting |
Widely used academic benchmarks include Market-1501, MSMT17 and the now-retired DukeMTMC-reID. Leading models report rank-1 accuracy above 95 percent on Market-1501 but considerably lower scores on MSMT17, and accuracy drops sharply in cross-domain evaluation, where a model trained in one camera network is tested in another.
Person re-identification compared with related tasks
Person re-identification overlaps with several neighboring computer vision tasks but differs in its open-set, cross-camera retrieval formulation. The distinctions clarify where ReID fits in a larger system.
- Facial recognition matches faces and needs a visible, reasonably large face; ReID works on full-body appearance at low resolution.
- Object tracking follows a target within one continuous view; ReID links identities across camera gaps and time gaps.
- Image classification assigns one of a fixed set of labels; ReID handles identities never seen during training.
- Content-based image retrieval is the general family to which ReID belongs, specialized for pedestrians.
In multi-camera tracking systems, ReID features are frequently combined with motion and timing constraints, because appearance alone is ambiguous when many people wear similar clothing.
Limitations, privacy and failure modes
Person re-identification fails most often under appearance ambiguity and domain shift, and it raises serious privacy and civil-liberty concerns because it enables following individuals across public spaces without their knowledge.
The table pairs typical symptoms with their cause and a common remedy.
| Symptom | Likely cause | Typical remedy |
|---|---|---|
| Wrong match in similar uniforms | Clothing dominates the embedding | Part-based or attribute-aware features |
| Accuracy collapse at a new site | Domain shift in lighting and cameras | Domain adaptation or target-site fine-tuning |
| Misses for partly hidden people | Occlusion removes key regions | Visibility-aware part matching |
| Poor results at night | Infrared and color images differ | Cross-modality training |
| Uneven accuracy across groups | Unbalanced training data | Dataset auditing and balanced sampling |
In the European Union, appearance-based tracking of identifiable people counts as processing of personal data under the GDPR, and the AI Act places strict limits on remote biometric identification in public spaces. Responsible deployments therefore apply purpose limitation, short retention and human review of every match.
Understanding it simply
Person re-identification answers a simple question: is the person in this camera the same person seen in another camera? It does this without reading a name or seeing a face clearly, by comparing overall appearance such as clothing, build and the bag someone carries.
It works a little like spotting a friend in a crowded station by their red coat and backpack. The system turns each picture into a numeric fingerprint and looks for the closest fingerprints. It can be very accurate, but it is easily fooled when people dress alike or change clothes.
Applied in practice
A large international airport operates about 400 cameras in its terminals. When a parent reports a lost seven-year-old near Gate B12, security staff upload a photo taken that morning at check-in. The ReID system compares it with embeddings from the last 30 minutes of footage across all terminal cameras.
Within seconds, the system returns a ranked list of 20 candidate sightings. An operator reviews the top results, confirms that ranks 1, 2 and 4 show the child walking toward a food court, and dispatches staff to the right location in under five minutes. Rank 3 is a different child in a similar jacket, which the operator discards.
Because the search is limited to an active incident, the airport keeps embeddings for only 72 hours and logs every query. Human confirmation of each match prevents a false identification from triggering action against the wrong family.