DiskANN

On this page 7

DiskANN is a graph-based approximate nearest neighbor (ANN) index designed to store most of its data on a solid-state drive (SSD) instead of in main memory, so that a single server can search billions of vectors with high recall and low latency. In the field of Retrieval & knowledge within artificial intelligence, it serves as the engine behind large semantic search and retrieval-augmented generation systems whose embedding collections are too large or too expensive to hold entirely in RAM.

The approach was introduced by Microsoft Research in 2019 together with its core graph construction algorithm, Vamana. Its central idea is simple: keep a compressed copy of every vector in memory to guide the search, keep the full-precision vectors and the graph on disk, and design the graph so that a query needs only a few dozen SSD reads to reach its nearest neighbors.

How DiskANN searches vectors stored on SSD

DiskANN answers a query by walking a proximity graph in which each vector is a node connected to a limited number of nearby nodes. The walk starts at a fixed entry point, usually a node close to the dataset centroid, and repeatedly moves toward neighbors that are closer to the query until no further improvement is found.

The distinctive part is the memory and disk layout. Each node’s full-precision vector and its adjacency list are stored together in one SSD sector of about 4 KB, so a single read returns both the exact vector and the list of neighbors to explore next. Distances used for steering are computed against product-quantized (PQ) codes held in RAM, which are cheap to evaluate but approximate.

To hide SSD latency, DiskANN uses beam search: in each step it fetches the adjacency lists of several promising candidates in parallel rather than one at a time. At the end, the exact vectors already read from disk are used to re-rank the candidates with full-precision distances, which recovers the accuracy lost to compression.

ComponentStorage locationRole in a query
PQ-compressed vectorsRAMFast approximate distances to steer the walk
Graph adjacency listsSSDDefine which nodes to visit next
Full-precision vectorsSSD, same sector as the nodeExact re-ranking of final candidates
Entry point and cached nodesRAMShorten the first hops of every search
Where each part of a DiskANN index lives

The Vamana graph and its build parameters

Vamana is the algorithm that builds the DiskANN graph, and its purpose is to create a flat, single-layer graph with a small diameter so that searches converge in few hops. It starts from a random graph and refines it in two passes, each time running a greedy search for every point and pruning the resulting candidate neighbors.

The key innovation is alpha-based pruning. A standard relative-neighborhood rule discards a candidate neighbor when another selected neighbor is closer to it; Vamana relaxes this rule with a factor alpha greater than 1, which keeps some long-range edges. These long edges let a search cross the dataset quickly, which matters more on disk than in memory because every hop costs an SSD read.

The following table lists the parameters practitioners tune most often and the typical ranges used in published configurations and vector databases.

ParameterTypical rangeEffect of a higher value
R (maximum degree)32 to 128Better recall, larger index, slower build
L build (candidate list size)75 to 200Higher graph quality, longer build time
Alpha (pruning factor)1.0 to 1.4More long-range edges, fewer hops per query
L search (search list size)50 to 300Higher recall, more SSD reads, higher latency
Beam width W2 to 8Fewer round trips, more reads per step
PQ code size16 to 128 bytes per vectorMore accurate steering, more RAM used
Main DiskANN build and search parameters

DiskANN differs from other ANN methods mainly in where the index lives and therefore in its cost profile. HNSW builds a multi-layer graph that performs excellently but expects all vectors and edges in RAM. An IVF index partitions vectors into clusters and scans only a few of them, which works on disk but usually needs more reads per query for the same recall. Exact nearest neighbor search compares the query with every vector and becomes impractical at large scale.

MethodPrimary storageStrengthMain limitation
DiskANNSSD plus compressed RAMBillion-scale search on one nodeDepends on fast NVMe SSD
HNSWRAMVery low latency and high recallMemory cost grows with full vectors
IVF with PQRAM or diskSimple, fast to buildRecall drops with heavy compression
Exact searchRAM or diskPerfect recallLinear cost per query
How DiskANN relates to neighboring index types

The practical consequence is memory economics. A billion 768-dimensional float32 embeddings occupy roughly 3 TB in full precision, while a DiskANN index can keep only compressed codes of about 32 to 64 GB in RAM and place the rest on SSD that costs far less per gigabyte.

Variants and use in retrieval systems

The original DiskANN assumed a static dataset, so several variants extend it for production retrieval. They address updates, filtering and integration with database engines, which are common needs in retrieval-augmented generation (RAG) pipelines.

  • FreshDiskANN supports inserts and deletes by buffering changes in an in-memory graph and merging them into the SSD index in the background.
  • Filtered-DiskANN builds label-aware edges so that metadata filtering does not break graph navigability.
  • In-memory Vamana uses the same graph without SSD and competes directly with HNSW.
  • Several vector databases and PostgreSQL extensions offer DiskANN-style indexes, sometimes with their own compression schemes.

Choosing DiskANN is mostly a question of scale, budget and latency target. The decision table below summarizes common situations.

SituationRecommended choiceReason
Under 10 million vectors, ample RAMHNSW or in-memory indexLowest latency, simple operation
Hundreds of millions to billions of vectorsDiskANN on NVMe SSDAvoids terabytes of RAM
Frequent inserts and deletesFreshDiskANN or database integrationStatic build would go stale
Strict tenant or label filtersFiltered-DiskANNKeeps recall under restrictive filters
Storage on network disks onlyIVF or managed serviceHigh disk latency hurts graph hops
When DiskANN is a sensible choice

Limitations and common mistakes

DiskANN performs well only when its hardware assumptions hold. It expects local NVMe SSDs with high random-read throughput; on network-attached or spinning disks, each hop can take milliseconds and query latency grows sharply. Index construction is also memory and time intensive, although the build can be split into overlapping partitions that are merged afterward.

SymptomLikely causeFix
Latency far above a few millisecondsSlow or shared storageMove the index to local NVMe
Recall below targetL search or PQ code too smallRaise L search or code size
Results ignore filtersPost-filtering after searchUse filter-aware indexing
Stale results after updatesStatic index never rebuiltUse a fresh index or rebuild schedule
Frequent problems with DiskANN deployments

A frequent misconception is that DiskANN is slower by nature because it uses disk. With a well-built graph and beam search, published benchmarks report over 90 percent recall at a few milliseconds per query on a single node, although results depend strongly on dataset, dimensionality and hardware. Another mistake is treating its recall as exact: like every ANN method, it trades a small accuracy loss for large gains in speed and cost.

Put simply

DiskANN is a way to find similar items in a huge collection without keeping the whole collection in a computer’s fast memory. It keeps a small summary of every item in memory and the detailed versions on a fast storage drive.

Think of a large library with a pocket map. The map in your hand is rough, but it tells you which shelves are worth visiting. You walk to only a handful of shelves, check the exact books there, and pick the best matches. The key benefit is that one ordinary server can search billions of items quickly and at a much lower cost than a memory-only system.

In practice

A legal research provider wants a RAG assistant over about 800 million text chunks, each stored as a 1,024-dimensional embedding. Keeping these vectors in RAM with an in-memory graph would require several terabytes of memory spread across many servers, which the team judges too costly.

The engineers build a DiskANN index with a maximum degree of 64, alpha set to 1.2 and 64-byte PQ codes, and place it on two local NVMe drives in a single server with 256 GB of RAM. At query time they use a search list size of 120 and a beam width of 4, then pass the top 50 candidates to a cross-encoder reranker.

In internal tests, the index reaches recall of about 95 percent at 10 results with median latency near 4 milliseconds. The assistant now answers questions from the full archive while the hardware cost falls to a fraction of the memory-only design.