SigLIP

Also known as: Sigmoid loss for language-image pre-training

On this page 7

SigLIP, short for sigmoid loss for language-image pre-training, is a method for training paired image and text encoders in which every image-caption combination in a batch is judged as an independent binary decision: matching or not matching. Introduced by Google researchers in 2023, it is a direct refinement of the CLIP recipe and has become one of the most widely used sources of vision encoders in multimodal AI.

The change looks small, because only the loss function differs, yet it removes the need for a global normalization across the batch. That makes training cheaper in memory, more stable at small batch sizes and easier to distribute across many accelerators, which explains why SigLIP image towers sit inside many current vision-language models.

How the sigmoid loss scores image-text pairs

SigLIP trains an image encoder and a text encoder so that embeddings of matching pairs end up close together and embeddings of unrelated pairs end up far apart. Both encoders are usually Vision Transformer and Transformer text models, and each produces a normalized vector in a shared embedding space.

For a batch of N pairs, the model computes an N by N similarity matrix of dot products. The N diagonal entries are positives and the remaining entries are negatives. Each entry is multiplied by a learnable temperature, shifted by a learnable bias, and passed through a logistic sigmoid with a binary label of plus one or minus one. The total loss is simply the sum of independent binary losses, with no row-wise or column-wise softmax.

The bias term matters more than it first appears. Because a batch contains roughly N positives against N squared negatives, the classifier would otherwise spend its early training pushing everything toward “no match”. The original recipe initializes the bias at about -10 and the log temperature at about log 10, so the model starts from a sensible prior and the heavy class imbalance does not destabilize learning.

The table summarizes the main components of the SigLIP objective and their role.

ComponentRoleTypical setting
Similarity matrixScores every image against every captionN by N dot products
Temperature tScales logits to sharpen or soften decisionsLearned, initialized near 10
Bias bOffsets the prior toward negativesLearned, initialized near -10
Sigmoid per pairTurns each logit into a match probabilityBinary label of +1 or -1
Summed lossCombines all pair losses without normalizationAveraged over batch size
Components of the SigLIP training objective

SigLIP compared with the CLIP softmax loss

The essential difference from CLIP is the shape of the normalization. CLIP applies a softmax over each row and each column of the similarity matrix, so the probability that an image matches its caption depends on every other caption in the batch. SigLIP evaluates each pair on its own, so a pair’s loss does not depend on the rest of the batch except through shared parameters.

This independence has practical consequences for distributed training. A softmax loss requires gathering all embeddings across devices to compute the normalizer, while the sigmoid loss can be computed in chunks that pass embeddings between devices one block at a time. The table compares the two approaches.

AspectCLIP softmax lossSigLIP sigmoid loss
NormalizationOver rows and columns of the batchNone, each pair is independent
Memory patternNeeds a full similarity matrixCan be computed block by block
Small batch behaviorDegrades noticeablyRemains comparatively strong
Extra parametersTemperature onlyTemperature and bias
Output meaningRelative ranking within a batchApproximate per-pair match probability
Softmax contrastive loss versus sigmoid loss

In the published experiments, the sigmoid loss outperformed the softmax loss clearly at batch sizes below about 16,000 and matched or slightly exceeded it at larger sizes. Both losses showed diminishing returns beyond roughly 32,000, and a batch of one million gave no meaningful further gain, which corrected a common belief that contrastive learning always benefits from ever larger batches.

Model variants and the SigLIP 2 family

SigLIP checkpoints are released in several encoder sizes and input resolutions, and the choice between them is a trade-off between accuracy, latency and memory. The best-known configuration is So400m, a shape-optimized Vision Transformer of about 400 million parameters whose width and depth were tuned for compute efficiency.

VariantApproximate image tower sizeTypical use
B/16About 86 million parametersFast retrieval and edge deployment
L/16About 300 million parametersHigher-accuracy retrieval and classification
So400m/14About 400 million parametersVision encoder for large vision-language models
Resolutions224, 256, 384 or 512 pixelsHigher values for documents and fine detail
Common SigLIP encoder configurations

SigLIP 2, released in 2025, keeps the sigmoid objective but adds a captioning-based decoder loss, self-distillation and masked prediction, which improve dense features and localization. It also trains on more multilingual data and offers a NaFlex variant that accepts images at their native aspect ratio, useful for documents and screenshots where squashing an image to a square destroys text.

Where SigLIP is used in practice

SigLIP serves two broad roles. As a complete dual encoder, it performs zero-shot image classification and cross-modal retrieval directly: class names or queries are embedded as text and compared with image embeddings. As a component, its image tower is detached and connected through a multimodal projector to a language model, as in PaliGemma and many open vision-language models.

The decision table maps common requirements to a sensible SigLIP choice.

SituationRecommended choiceReason
Image search over a product catalogFull dual encoder, B/16 or L/16Cheap embeddings, strong retrieval
Building a vision-language modelSo400m image tower at 384 pixelsRich patch features for the projector
Reading charts and documentsHigher resolution or NaFlex variantPreserves small text and layout
Non-English queriesMultilingual SigLIP 2 checkpointBroader language coverage in the text tower
Tight compute budgetLocked image tower with trained text towerFast convergence with few accelerators
Choosing a SigLIP setup for a task

The last row refers to SigLiT, a variant that freezes a pretrained image model and trains only the text side. The original paper reported strong ImageNet zero-shot accuracy with this setup using only four TPU chips for about two days, which illustrates how economical the sigmoid objective can be.

Limitations and common misconceptions

SigLIP inherits most weaknesses of web-scale image-text training. Captions scraped from the web are noisy, short and culturally skewed, so the model learns dataset biases and can misread counting, spatial relations and negation. These are properties of the data and the contrastive objective, not of the sigmoid loss itself.

  • Sigmoid scores are not calibrated probabilities out of the box; thresholds for a match must be tuned on held-out data for each application.
  • SigLIP is not a generative model; it scores and embeds but does not write captions unless paired with a decoder or language model.
  • Larger batches do not guarantee better models; gains flatten near 32,000 pairs, and data quality usually matters more.
  • Global embeddings lose fine detail; tasks that need precise localization benefit from patch features or the SigLIP 2 additions.
  • Fine-tuning can erode zero-shot ability; aggressive task-specific training often reduces general retrieval quality.

In plain words

SigLIP teaches a computer to tell whether a picture and a sentence belong together. It looks at many pictures and captions at once and, for every possible pairing, asks a simple yes-or-no question: does this caption describe this picture?

Older methods such as CLIP made each picture compete against all captions in the group, a bit like a multiple-choice test. SigLIP grades every pairing separately, like a stack of true-or-false questions. The key takeaway is simple: this makes training lighter and steadier, and the resulting image reader is good enough that many chat assistants that understand photos use it as their eyes.

A real-world example

A mid-sized online furniture retailer wants customers to search its catalog of about 200,000 product photos with free-text queries such as “walnut desk with drawers”. The engineering team embeds every image once with a SigLIP L/16 model at 384 pixels and stores the vectors in a vector database, which takes a few hours on a single GPU.

At query time, only the short text query is embedded, and the nearest image vectors are returned in milliseconds. Because sigmoid scores behave roughly like per-pair match probabilities, the team tunes a single relevance threshold on a few hundred labeled queries to hide weak matches. In offline testing, the share of searches that end with a click on a result rises noticeably compared with the old keyword search, and no product needed manual tagging to become searchable.