An IVF index, or inverted file index, is a vector index that partitions a collection of embeddings into clusters around learned centroids and, at query time, compares the query only with the vectors stored in the few clusters closest to it. In the Retrieval & knowledge part of artificial intelligence, it is one of the most widely used structures for approximate nearest neighbor search, because it turns a scan of millions of vectors into a scan of a small, carefully chosen fraction.
The name comes from classic text retrieval, where an inverted file maps each word to the documents that contain it. An IVF index applies the same idea to vector space: each centroid acts as a list key, and its posting list holds the identifiers and vectors assigned to that region. The result is a tunable trade-off between recall and latency that suits large semantic search and RAG workloads.
How an IVF index partitions vector space
An IVF index works in two phases: a training phase that learns the partition and a query phase that searches only part of it. During training, a clustering algorithm, usually k-means, runs on a representative sample of vectors and produces a set of centroids, often called the coarse quantizer. Each centroid defines a cell, and together the cells form a Voronoi partition of the embedding space.
During ingestion, every vector is assigned to its nearest centroid and appended to that centroid’s inverted list. At query time, the engine compares the query with all centroids, selects the nprobe closest cells, and computes exact or compressed distances only for the vectors inside those cells. The final candidates are sorted by distance and the top k are returned.
The following table summarizes the stages of the pipeline and the cost each one carries.
| Stage | What happens | Main cost |
|---|---|---|
| Training | k-means learns nlist centroids from a sample | Compute time, once per rebuild |
| Assignment | Each vector joins its nearest centroid list | One centroid comparison per vector |
| Coarse search | Query is compared with all centroids | Grows with nlist |
| Fine search | Vectors in nprobe lists are scored | Grows with nprobe and list size |
| Ranking | Candidates are sorted and top k returned | Small, proportional to candidates |
Key parameters: nlist, nprobe and training data
Two parameters dominate IVF behavior: nlist, the number of clusters, and nprobe, the number of clusters searched per query. A widely used starting rule sets nlist near the square root of the collection size, with values between roughly 4 times and 16 times that root for very large sets. For one million vectors, an nlist of 1,024 to 4,096 is typical.
The nprobe value controls recall directly. Searching 1 percent to 10 percent of the lists often reaches recall@10 above 0.9 on well-clustered embeddings, but the exact figure depends on the data distribution and the embedding model. Training data matters as well: the sample should hold at least 30 to 100 vectors per centroid and reflect the real distribution, or the clusters will be unbalanced.
The table below lists the main settings, their usual ranges and their effect on results.
| Parameter | Typical range | Effect when increased |
|---|---|---|
| nlist | Square root of N up to 16 times that | Smaller lists, faster fine search, more centroid work |
| nprobe | 1 to 10 percent of nlist | Higher recall, higher latency |
| Training sample | 30 to 256 vectors per centroid | More stable, balanced clusters |
| Code size with PQ | 8 to 64 bytes per vector | Better accuracy, more memory |
| Refine factor | 2 to 10 times k | More exact reranking, slower queries |
IVF variants and compression
The term IVF describes the partitioning layer, and it is usually combined with a vector encoding that decides how list entries are stored. IVF-Flat keeps full-precision vectors and gives the best accuracy per probed list. IVF-PQ stores vectors as product quantization codes, cutting memory by a factor of 10 to 60, while IVF-SQ8 uses 8-bit scalar quantization as a middle ground.
Most IVF-PQ deployments encode the residual vector, the difference between a vector and its centroid, because residuals have a smaller spread and quantize more accurately. A common pattern adds a reranking step that reloads full vectors for the best few hundred candidates to recover the precision lost through compression.
| Variant | Stored data | Best suited to |
|---|---|---|
| IVF-Flat | Full float32 vectors | High recall, moderate collections |
| IVF-SQ8 | 8-bit scalar codes | Four times less memory, small accuracy loss |
| IVF-PQ | Product quantization codes | Billions of vectors, tight memory |
| IVF-HNSW coarse | Centroids searched with a graph | Very large nlist values |
| IVF on GPU | Flat or PQ lists in GPU memory | High query throughput in batches |
IVF compared with HNSW and other vector indexes
IVF is one of several families of approximate nearest neighbor methods, and it differs mainly in memory profile and update behavior. HNSW, a graph-based index, typically offers lower latency at high recall but stores many neighbor links per vector and uses more memory. DiskANN keeps a graph on SSD for collections that exceed RAM, while exact nearest neighbor search scans everything and serves as the accuracy baseline.
| Method | Memory use | Strength | Weakness |
|---|---|---|---|
| IVF index | Low to moderate | Compact, combines well with PQ | Needs training, recall depends on nprobe |
| HNSW | High | Fast, high recall without training | Large graph overhead |
| DiskANN | Low RAM, SSD heavy | Billion-scale on one machine | Disk latency, complex builds |
| Exact search | Raw vectors only | Perfect recall | Linear cost per query |
In practice, IVF is preferred when the collection is large and memory is the binding constraint, when GPU batch search is available, or when filters can be mapped onto partitions. HNSW is often chosen for collections that fit comfortably in RAM and need very low single-query latency.
Limitations and common mistakes with IVF
The most frequent problem is distribution drift: centroids trained on old data fit new embeddings poorly, lists grow unevenly, and recall falls without any visible error. A related issue is boundary loss, where a true neighbor lies in an adjacent cell that was not probed. Both effects are measured, not guessed, by comparing results with an exact baseline.
- Training on too few vectors produces empty or oversized lists and unstable recall.
- Keeping nprobe at 1 gives impressive speed and disappointing recall on real queries.
- Ignoring metric consistency breaks results, for example training with L2 distance while serving cosine similarity on unnormalized vectors.
- Strict metadata filtering after the search can empty the result set when the probed lists contain few matching items.
- Changing the embedding model without retraining invalidates every centroid.
Periodic retraining and reindexing on a schedule tied to data growth, together with recall monitoring on a fixed query set, keeps an IVF index reliable over time.
A non-technical view
Imagine a huge library where every book has been placed in one of a thousand themed rooms. When you look for books similar to the one in your hand, you do not walk through every room. You check which few rooms match your book best and search only there. That is what an IVF index does with numbers that describe text, images or other data.
The method is fast because it skips most rooms, but occasionally the best match sits in a room you did not open. Searching more rooms finds more of the right answers but takes longer, so the key decision is how many rooms to check.
A concrete example
An internal support team builds a RAG assistant over 40 million document chunks, each represented by a 768-dimensional embedding. Stored as full float32 vectors, the collection would need about 120 GB of memory, which exceeds the budget of a single search server.
The engineers build an IVF-PQ index with an nlist of 16,384, train it on two million sampled vectors, and encode each vector in 64 bytes, bringing the index to roughly 3 GB including identifiers. With nprobe set to 64 and a reranking step over the top 200 candidates using full vectors stored on SSD, they measure recall@10 of about 0.94 against an exact baseline. Median search latency stays near 12 milliseconds, and a monthly retraining job keeps recall stable as new documents arrive.