A vector database is a database system designed to store high-dimensional vectors, usually embeddings, and retrieve the vectors most similar to a query vector with low latency at large scale. Among database models, it is defined by its primary query: not exact matching on keys or values, but similarity search under a distance metric such as cosine distance or Euclidean distance.
Embeddings are numeric representations produced by machine learning models, in which semantically similar items lie close together. A vector database makes that geometry searchable, which is why it underpins semantic search, recommendation and retrieval-augmented generation. The capability exists both as dedicated products and as extensions of relational, document and search engines.
How a vector database finds similar items
A vector database finds similar items by computing distances between the query vector and stored vectors and returning the k closest, a k-nearest neighbor (k-NN) query. Exact search compares the query with every stored vector, which is precise but costs time proportional to the collection size and the dimension, typically 384 to 3,072 dimensions for modern text embeddings.
At millions or billions of vectors, exact search becomes too slow, so vector databases use approximate nearest neighbor (ANN) indexes. These indexes give up a small amount of recall, the share of true nearest neighbors returned, in exchange for answering in milliseconds. Recall of 0.90 to 0.99 is a common operating target, and it is tuned by index parameters at build and query time.
| Metric | What it measures | Typical use |
|---|---|---|
| Cosine distance | Angle between vectors, ignoring length | Text embeddings |
| Dot product | Alignment including magnitude | Normalized embeddings, recommendation |
| Euclidean (L2) distance | Straight-line distance | Image and audio features |
| Hamming distance | Differing bits | Binary quantized vectors |
The metric must match how the embedding model was trained. Using Euclidean distance on embeddings meant for cosine similarity can noticeably reduce result quality, although for unit-length vectors the rankings of cosine, dot product and Euclidean distance coincide.
Approximate nearest neighbor index types
The index type determines a vector database’s speed, memory footprint, recall and update behavior. Most systems offer several, and the choice depends on data volume and hardware.
| Index | Mechanism | Strength | Weakness |
|---|---|---|---|
| HNSW | Multi-layer proximity graph navigated greedily | High recall at low latency | High memory use, slow builds |
| IVF | Vectors clustered; only nearby clusters searched | Lower memory, fast builds | Recall depends on clusters probed |
| Product quantization | Vectors compressed into short codes | Fits billions of vectors in memory | Lossy distances reduce recall |
| Disk-based graph | Graph index stored mostly on SSD | Large collections on modest RAM | Higher latency than in-memory |
Key tuning parameters include the graph degree and the search breadth in HNSW, often called M and ef, and the number of probed clusters in IVF. Raising them improves recall and increases latency and memory, so they are set by measuring recall against an exact baseline on a sample of real queries.
Vector database compared with other database models
A vector database differs from neighboring database models in what a query asks for: closeness in meaning rather than equality, structure or time order.
| Model | Core query | Typical index | Typical workload |
|---|---|---|---|
| Vector database | Top-k most similar vectors | HNSW, IVF, quantization | Semantic search, recommendations |
| Document database | Match fields in JSON documents | B-tree on fields | Application data, catalogs |
| Graph database | Traverse relationships | Adjacency structures | Networks, fraud paths |
| Time-series database | Aggregate values over time | Time-partitioned columns | Metrics, sensor data |
| Relational with vector extension | SQL plus similarity operator | ANN index beside B-trees | Moderate scale, joined filters |
Real applications rarely search vectors alone. Filtered vector search combines similarity with conditions such as tenant, language or date, and hybrid search merges vector results with keyword ranking. How well a system handles selective filters is often more decisive than its raw similarity speed.
Choosing and operating a vector database
The right setup depends on collection size, filter needs and update rate. Smaller collections rarely require a dedicated system.
| Situation | Reasonable choice | Reason |
|---|---|---|
| Up to a few million vectors, existing SQL data | Vector extension in the current database | One system, transactional consistency |
| Hundreds of millions of vectors | Dedicated distributed vector database | Sharded ANN indexes and replication |
| Strict memory budget | IVF with quantization or disk-based index | Lower RAM per vector |
| Frequent deletes and updates | Index with incremental updates and compaction | Avoids recall decay from stale entries |
Memory planning is concrete arithmetic: a float32 vector of 1,536 dimensions occupies about 6 KB, so ten million such vectors need about 60 GB before index overhead. Quantization to 8-bit integers cuts this roughly fourfold, at some cost in accuracy.
Limitations and common mistakes
Vector search fails quietly: it always returns the nearest results, even when none of them is relevant. The usual mistakes follow from that property.
- Mixing embeddings from different models or versions in one index, which makes distances meaningless.
- Skipping a relevance threshold, so irrelevant neighbors reach users or language models.
- Measuring speed without measuring recall, which hides poor result quality.
- Embedding long documents whole instead of chunking them into passages.
The simple explanation
A vector database is a search engine for similarity. Instead of looking for exact words, it turns texts, pictures or products into lists of numbers. Things with similar meaning get similar numbers.
Imagine a huge map where every song is a dot. Songs that sound alike sit close together. When you ask for something like a given song, the database looks around that spot on the map and returns the nearest dots. It does this quickly, even with millions of songs.
How it is used in practice
A software company runs a support assistant over 400,000 help articles and resolved tickets. Each document is split into passages of a few hundred words and embedded into 768-dimensional vectors, about 2.5 million in total, stored in an HNSW index with filters for product and language.
When a customer asks a question, the system embeds it, retrieves the 20 most similar passages within the customer’s product, and passes them to a language model that drafts an answer. Queries return in about 30 milliseconds at a measured recall of 0.95. Because results below a similarity threshold are discarded, the assistant escalates to a human instead of answering when no relevant passage exists.