Facial recognition is a computer vision technique that identifies or verifies a person by converting an image of their face into a numerical representation and comparing it with stored representations of known faces. In computer vision, also called face recognition, it is a specialized form of biometric matching built on face detection, alignment and deep metric learning.
Facial recognition answers either “is this the claimed person?” or “who is this person?”, and the difference between those two questions shapes its accuracy, its risks and its legal treatment. It should not be confused with face detection, which only finds faces, or with facial analysis, which estimates attributes such as expression.
How facial recognition works
A modern facial recognition system is a pipeline of four stages. Each stage can fail independently, so overall accuracy depends on the weakest link, frequently the quality of the input image rather than the recognition model itself.
| Stage | What happens | Typical technique |
|---|---|---|
| Face detection | Locates faces with a bounding box | CNN or transformer detectors |
| Alignment | Normalizes pose, scale and rotation | Keypoint detection of eyes, nose, mouth |
| Embedding | Encodes the face as a feature vector | Deep CNN trained with margin-based loss |
| Matching | Compares vectors against templates | Cosine similarity with a threshold |
The central idea is the face embedding, a vector typically of 128 to 512 dimensions. The network is trained so that embeddings of the same person lie close together and embeddings of different people lie far apart. Loss functions such as triplet loss and additive angular margin losses enforce this geometry during training on millions of labeled face images.
At decision time the system computes a similarity score between two embeddings and compares it with a threshold. Raising the threshold reduces false matches but increases false rejections, so the threshold is the most important operational setting in any deployment.
Verification and identification
Facial recognition operates in two modes that differ sharply in difficulty. Verification is a one-to-one comparison, such as unlocking a phone. Identification is a one-to-many search against a gallery, such as finding a person in a database of thousands or millions of faces.
| Property | Verification (1:1) | Identification (1:N) |
|---|---|---|
| Question answered | Is this the claimed person | Who is this person |
| Comparisons per query | One | Up to the size of the gallery |
| Error growth | Stable | False matches rise with gallery size |
| Typical use | Device unlock, border gates | Investigations, watchlists |
| Consent context | Usually voluntary | Often without the subject’s knowledge |
The key technical consequence is that false match probability multiplies with gallery size. A false match rate that is negligible in a one-to-one check can produce several false candidates per search in a large database, which is why identification results should be treated as leads for human review, never as conclusions.
Accuracy metrics and influencing factors
Facial recognition accuracy is measured with two linked error rates rather than a single accuracy figure. The false match rate (FMR) is the share of impostor comparisons wrongly accepted; the false non-match rate (FNMR) is the share of genuine comparisons wrongly rejected. Results are reported as FNMR at a fixed FMR, for example at one in a million, following biometric testing standards such as ISO/IEC 19795.
Leading algorithms achieve very low error rates on cooperative, frontal, well-lit images, but performance drops under real conditions. The following factors account for most practical failures.
| Factor | Effect | Common mitigation |
|---|---|---|
| Pose and angle | Fewer visible features | Multi-view enrollment, alignment |
| Low resolution | Loss of fine detail | Minimum pixel distance between eyes |
| Lighting and blur | Noisy embeddings | Image quality scoring and rejection |
| Occlusion | Masks, glasses, hands hide regions | Occlusion-aware training data |
| Aging | Appearance drifts over years | Periodic re-enrollment |
| Demographic imbalance | Unequal error rates across groups | Balanced training and per-group testing |
Independent evaluations have documented demographic differentials, where error rates vary with age, sex and skin tone. The size of these gaps differs greatly between algorithms, so each system must be tested on the population where it will actually be used.
Spoofing and liveness detection
A recognition model compares appearance; it does not know whether a live person is present. Presentation attacks use printed photos, screen replays, masks or deepfake video to impersonate someone. Liveness detection, standardized as presentation attack detection in ISO/IEC 30107, adds checks such as depth sensing, infrared imaging, texture analysis or challenge-response movements.
Systems for payments or identity onboarding generally combine recognition with liveness checks, because a high-accuracy matcher without anti-spoofing is easy to defeat with a simple photograph.
Liveness methods fall into two groups. Active liveness asks the user to blink, turn the head or follow a moving dot, which is robust but adds friction. Passive liveness analyzes a single capture for depth cues, skin texture and screen artifacts without any user action. Injection attacks, where a synthetic video stream is fed directly into the camera pipeline, require additional device and stream integrity checks beyond image analysis.
Privacy, regulation and responsible use
Face templates are biometric data, which many jurisdictions classify as a special category requiring a clear legal basis, strong security and often explicit consent. The European Union’s AI Act places remote biometric identification among its most restricted uses, with real-time use in publicly accessible spaces by law enforcement prohibited except in narrowly defined cases.
- Store templates, not images, and encrypt them at rest.
- Define the threshold deliberately and document the error rates it implies.
- Keep a human in the decision for any consequential identification.
- Test per demographic group before and during deployment.
- Limit retention and purpose, deleting templates when no longer needed.
A non-technical view
Facial recognition turns a face into a list of numbers that describes its shape, a little like a fingerprint made of measurements. To recognize someone, the system compares the numbers from a new photo with numbers it has stored and checks how close they are.
Confirming that a person is who they claim to be is relatively easy and reliable. Picking one person out of a huge crowd is much harder and more error-prone, so a match should start an inquiry, not end it.
A concrete example
An international airport introduces automated boarding gates. At check-in, each passenger consents to having a face template created from the photo in their passport chip. At the gate, a camera captures a frontal image and the system performs a one-to-one verification against that single template, with a threshold tuned for a very low false match rate.
A liveness check with an infrared camera rejects printed photos. Passengers who fail verification, often because of poor lighting or a large change in appearance, are directed to a staffed desk. Boarding becomes faster, templates are deleted within 24 hours of departure, and the operator publishes error rates measured on its own passenger population.