A Siamese network is a neural network architecture that passes two or more inputs through identical subnetworks with shared weights and compares the resulting embeddings to decide how similar the inputs are. Within neural networks, it is the classic design for similarity learning, where the goal is a meaningful distance between items rather than a fixed class label.
Because the network learns a general notion of similarity, it can compare items it never saw during training. This property made Siamese designs central to signature verification, face verification, one-shot image recognition and the sentence embedding models used in modern semantic search.
How a Siamese network compares two inputs
A Siamese network works by applying one encoder twice: the same weights map input A and input B to vectors in a common embedding space. A distance or similarity function, such as Euclidean distance or cosine similarity, then measures how close the two vectors are, and a threshold or a small head turns that score into a decision.
Weight sharing is the defining feature. It guarantees that identical inputs produce identical embeddings, halves the parameter count compared with two separate encoders and makes the comparison symmetric, so swapping A and B does not change the score. The encoder itself can be any backbone: a convolutional neural network for images, a recurrent network or transformer for text, or a graph neural network for molecules.
| Element | Role | Common choices |
|---|---|---|
| Shared encoder | Maps each input to an embedding | ResNet, transformer, LSTM |
| Embedding space | Holds comparable vectors | 64 to 1,024 dimensions, often normalized |
| Similarity function | Scores a pair of embeddings | Cosine similarity, Euclidean or L1 distance |
| Decision rule | Converts the score into an answer | Threshold, nearest neighbor, small classifier |
Loss functions used to train Siamese networks
A Siamese network is trained with a metric learning loss that pulls matching items together and pushes non-matching items apart. The choice of loss determines how training pairs are formed and how stable optimization is.
| Loss | Training unit | What it enforces |
|---|---|---|
| Contrastive loss | Pair labeled same or different | Small distance for matches, distance above a margin for non-matches |
| Triplet loss | Anchor, positive, negative | Anchor closer to the positive than to the negative by a margin |
| Binary cross-entropy | Pair with a match label | A head predicts the probability that inputs match |
| InfoNCE or in-batch softmax | One positive among many negatives | Correct pair scores highest within the batch |
The margin is a key hyperparameter: if it is too small, embeddings collapse together; if it is too large, training struggles to satisfy it. Hard negative mining, which selects non-matching examples that the model currently confuses, usually speeds learning considerably, but mining only the hardest cases can destabilize training, so semi-hard negatives are a common compromise.
Siamese networks compared with related architectures
A Siamese network differs from a standard classifier in what it outputs: a classifier assigns one of a fixed set of labels, whereas a Siamese network outputs a similarity score that works for classes unseen in training. This makes it suitable for open-ended identity problems where new people, products or documents appear constantly.
| Architecture | Input handling | Best suited for |
|---|---|---|
| Siamese network | Separate encoding with shared weights | Fast retrieval and verification at scale |
| Cross-encoder | Both inputs processed jointly | High-accuracy reranking of few candidates |
| Pseudo-Siamese network | Two encoders with separate weights | Inputs of different types, such as text and image |
| Standard classifier | One input, fixed label set | Closed sets of known categories |
The trade-off against a cross-encoder is central in practice. Because a Siamese model encodes each item independently, embeddings can be precomputed and indexed, and millions of items can be searched in milliseconds. A cross-encoder lets attention operate across both inputs and is usually more accurate, but it must run once per pair, so it is typically used only to rerank the top results that a Siamese bi-encoder retrieved. Dual-encoder models that align images and text follow the pseudo-Siamese pattern.
Applications and practical design choices
Siamese networks are used wherever a system must answer whether two items match or find the closest items in a large collection. Typical applications include:
- Face and signature verification, where a new sample is compared with an enrolled reference.
- One-shot and few-shot learning, where a single example per class is enough to classify new inputs.
- Semantic text search with sentence embeddings, including duplicate question detection.
- Image retrieval and product matching in catalogs with millions of entries.
- Change detection in remote sensing, comparing two images of the same area taken at different times.
In deployment, the verification threshold must be calibrated on held-out pairs to balance false accepts against false rejects, and normalizing embeddings to unit length makes cosine and Euclidean comparisons consistent. Approximate nearest neighbor indexes allow the embeddings to scale to very large collections.
Limitations and common mistakes
The most serious failure is embedding collapse, in which the encoder maps every input to nearly the same vector and every pair looks similar. It appears when negatives are missing, too easy or poorly sampled, and it is detected by monitoring the spread of embedding distances during training.
| Problem | Cause | Remedy |
|---|---|---|
| Collapsed embeddings | Too few or trivial negatives | Harder negatives, larger batches |
| Good training loss, poor retrieval | Pairs unlike real queries | Sample pairs from the deployment distribution |
| Unstable threshold | Score drift across domains | Recalibrate per domain on labeled pairs |
| Biased verification errors | Unbalanced training identities | Audit error rates per subgroup |
A further limitation is that independent encoding loses fine interactions between inputs, which is why Siamese models trail cross-encoders on subtle tasks such as detecting contradiction between sentences. In biometric uses, uneven error rates across demographic groups are a documented risk that requires careful data selection and testing.
In simple terms
A Siamese network is like one trained eye looking at two photos and saying how alike they are. It turns each item into a kind of fingerprint made of numbers, using exactly the same method for both, and then compares the fingerprints.
Because it learns what makes things similar rather than memorizing a fixed list of names, it can compare things it has never seen before. That is why a phone can recognize a new owner’s face after seeing it only a few times.
A practical example
An online marketplace with about 3 million product listings wants to detect sellers who post the same item several times under different titles. The engineering team trains a Siamese text and image encoder on 200,000 pairs of listings that moderators had already labeled as duplicates or distinct, using triplet loss with semi-hard negatives.
Every listing is encoded once into a 256-dimensional vector and stored in an approximate nearest neighbor index. When a new listing arrives, the system retrieves its 20 closest neighbors in a few milliseconds and flags pairs with cosine similarity above a calibrated threshold of 0.87 for review. The moderation team reports that far fewer duplicates reach buyers, while a cross-encoder rechecks only the flagged pairs to keep false alarms low.