SPLADE (Sparse Lexical and Expansion model) is a learned sparse retrieval model that uses a transformer language model to turn a query or document into a weighted bag of vocabulary terms, including relevant terms that never appear in the original text. In the retrieval & knowledge layer of artificial intelligence systems, it sits between classic keyword ranking such as BM25 and dense retrieval with embeddings: the output is a sparse vector that a standard inverted index can store and search, yet the term weights are learned from relevance data.
The model was introduced by researchers at Naver Labs Europe in 2021 and refined in several later versions. Its appeal is practical: it delivers neural-quality relevance while keeping the interpretability and infrastructure of lexical search, which makes it a common component of hybrid search and retrieval-augmented generation pipelines.
How SPLADE builds sparse term vectors
SPLADE produces each vector by passing text through a BERT-style encoder and reading the masked language modeling head, which scores every token position against the entire vocabulary of roughly 30,000 WordPiece tokens. Each score indicates how plausible a vocabulary term is at that position, so the head naturally proposes synonyms and related words.
The model then aggregates these scores into one weight per vocabulary term. It applies a ReLU activation to remove negative values, a logarithmic saturation to keep a few strong terms from dominating, and max pooling across positions (the first version used sum pooling). The result is a vector the size of the vocabulary in which most entries are zero.
Sparsity is not automatic. During training, a FLOPS regularizer penalizes the expected number of floating-point operations in index lookups, which pushes rarely useful weights to zero. The strength of this penalty is the main efficiency knob: a stronger penalty yields shorter vectors and faster search at some cost in effectiveness.
The following table summarizes the processing stages and what each contributes.
| Stage | Operation | Effect on the vector |
|---|---|---|
| Tokenization | WordPiece split of the input | Defines the term space |
| MLM head scoring | Logits over the full vocabulary per token | Proposes expansion terms |
| Activation and saturation | log(1 + ReLU(logit)) | Removes negatives, dampens peaks |
| Pooling | Max over token positions | One weight per vocabulary term |
| Sparsity regularization | FLOPS penalty during training | Most weights become zero |
Term weighting and term expansion
SPLADE solves two long-standing weaknesses of lexical retrieval at once. Term weighting replaces fixed statistics such as TF-IDF with context-aware importance, so a word that is central to the meaning of a passage receives a high weight even if it appears only once.
Term expansion addresses the vocabulary mismatch problem, where a relevant document uses different words than the query. A passage about a “heart attack” can receive nonzero weights for “myocardial” and “infarction”, so it matches a clinical query without any manual synonym list. Expansion applies to documents at indexing time and, in the standard configuration, to queries at search time.
Typical vector sizes depend on the regularization strength and the text length. The values below are indicative ranges rather than fixed properties of the model.
| Vector type | Typical nonzero terms | Main consequence |
|---|---|---|
| Document vector | About 100-300 for a passage | Determines index size |
| Query vector | About 10-60 | Determines lookup cost |
| BM25 query for comparison | Only the typed words | Fast but literal matching |
SPLADE compared with BM25 and dense retrieval
SPLADE belongs to the family of sparse retrieval methods but is trained like a neural ranker. Compared with BM25, it usually improves recall and ranking quality markedly on in-domain benchmarks such as MS MARCO. Compared with dense retrieval using a bi-encoder, it reaches similar in-domain effectiveness and often shows stronger zero-shot generalization on heterogeneous benchmarks such as BEIR, because exact lexical signals remain available.
The comparison below places SPLADE among its closest neighbors in the retrieval stack.
| Method | Representation | Index type | Key strength |
|---|---|---|---|
| BM25 | Raw term statistics | Inverted index | No training, very fast |
| SPLADE | Learned weights with expansion | Inverted index | Interpretable neural relevance |
| Dense bi-encoder | One continuous embedding | Vector index such as HNSW | Strong semantic matching |
| ColBERT | One embedding per token | Multi-vector index | Fine-grained late interaction |
SPLADE is not a reranker. It is a first-stage retriever that selects candidates from the full collection; a cross-encoder can still rerank its top results for additional precision.
Variants and deployment choices
Several SPLADE variants trade effectiveness against cost. SPLADE++ adds knowledge distillation from a cross-encoder teacher and training on hard negatives, which raised effectiveness considerably. Document-only SPLADE encodes only the documents and uses plain tokenized queries, which removes the neural encoder from the query path entirely.
The right configuration depends on latency budgets and hardware. The decision table below covers common situations.
| Situation | Recommended choice | Reason |
|---|---|---|
| Strict query latency, no GPU | Document-only encoding | No transformer at query time |
| Maximum first-stage quality | Distilled SPLADE++ model | Best learned relevance signal |
| Existing Lucene-based search | Index weights as term impacts | Reuses current infrastructure |
| Mixed lexical and semantic needs | Fuse with a dense retriever | Errors of the two models differ |
In practice, document weights are often quantized to integers and stored as term impacts in engines such as Lucene, Elasticsearch, OpenSearch or PISA. Results from SPLADE and a dense model are frequently combined with reciprocal rank fusion in hybrid search.
Limitations and common mistakes
SPLADE inherits costs from both worlds. Expanded documents create longer posting lists than BM25, so naive query processing can be several times slower than keyword search unless the vectors are well regularized and the engine uses dynamic pruning. Encoding a large corpus also requires GPU inference at indexing time.
- Vocabulary lock-in: the model can only express terms in its tokenizer vocabulary, which limits rare names, codes and non-English text unless a multilingual model is used.
- Domain shift: a model trained on web questions may weight specialized jargon poorly, so fine-tuning on in-domain pairs is often worthwhile.
- Overreading interpretability: expansion terms are inspectable, but their weights reflect training statistics, not verified semantic truth.
- Long documents: the encoder input limit of about 512 tokens means long texts must be split by chunking before encoding.
Stripped to the essentials
Classic search engines look for the exact words you typed. SPLADE adds a smart step before that: a language model reads each document and writes down the words it is really about, including helpful words the author never used, with a score for how important each one is.
Search still works the fast, old-fashioned way, by looking words up in an index, like the index at the back of a book. The difference is that this index was written by a reader who understands meaning. The key takeaway: SPLADE gives keyword search some of the understanding of AI without giving up its speed and transparency.
Where it is applied
A legal technology team runs search over about four million court decisions and supports a retrieval-augmented generation assistant for lawyers. Its BM25 system misses decisions that describe the same concept in different wording, while a dense-only prototype misses exact statute numbers and party names.
The team encodes the corpus once with a SPLADE++ model on two GPUs, stores the weights as integer impacts in its existing search cluster, and fuses SPLADE results with dense results using reciprocal rank fusion. A cross-encoder reranks the top 50 candidates before passages reach the language model.
In internal evaluation on a few hundred annotated queries, recall at 100 improved noticeably over BM25, exact citations continued to match, and query latency stayed within the existing budget. The outcome was a hybrid pipeline with fewer missed precedents and no new vector database for the lexical side.