Embeddings are dense numeric vectors that represent the meaning of data, such as a sentence, a document chunk, an image or a product, so that items with similar meaning lie close together in a shared vector space. In artificial intelligence, and especially in retrieval and knowledge systems, embeddings are the foundation of semantic search, retrieval-augmented generation, clustering and recommendation.
An embedding model turns each input into a fixed-length list of numbers, typically a few hundred to a few thousand dimensions. Comparing two such vectors with a similarity measure gives a fast, approximate answer to the question of how related the two inputs are, even when they share no words.
How embeddings capture meaning
An embedding model is a neural network, usually a transformer encoder, trained so that related inputs produce nearby vectors and unrelated inputs produce distant ones. The most common training method is contrastive learning: the model sees pairs that belong together, such as a question and its answer, and is pushed to place them close while separating them from many unrelated examples in the same batch.
After training, each dimension has no fixed human meaning on its own. Meaning is encoded in the overall direction of the vector, which is why the same model must be used for both the stored documents and the incoming queries. Vectors from different models are not comparable, even when their dimensions match.
The table summarizes the parameters that define an embedding setup and the values commonly encountered.
| Parameter | Typical values | Effect |
|---|---|---|
| Dimensions | 256 to 4,096 | Expressiveness versus storage and speed |
| Maximum input length | 512 to about 8,000 tokens | How much text fits in one vector |
| Numeric precision | 32-bit float, 8-bit, binary | Memory footprint and slight accuracy loss |
| Normalization | Unit length or raw | Whether dot product equals cosine |
| Model scope | General, domain, multilingual | Quality on specific content |
Measuring similarity between embeddings
Two embeddings are compared with a vector similarity function. The choice should match how the model was trained, which model documentation normally states.
| Measure | What it compares | Notes |
|---|---|---|
| Cosine similarity | Angle between vectors | Range -1 to 1, ignores length |
| Dot product | Angle and magnitude together | Equals cosine for normalized vectors |
| Euclidean distance | Straight-line distance | Smaller means more similar |
Similarity scores are relative, not absolute. A cosine of 0.8 may mean strong relevance for one model and mediocre relevance for another, so fixed thresholds must be calibrated on real data rather than copied between systems.
At scale, exact comparison against millions of vectors is too slow, so vector databases use approximate nearest neighbor indexes such as HNSW graphs or inverted file indexes. These trade a small amount of recall for search times in milliseconds.
Embeddings in a retrieval pipeline
In a RAG system, embeddings connect a user question to the passages that can answer it. Documents are first split by chunking into passages, each passage is embedded and stored, and at query time the question is embedded with the same model and matched against the stored vectors.
Embedding retrieval is fast but coarse, so production systems often add a reranking stage. A cross-encoder reranker reads the query and each candidate together, which is slower but considerably more precise, and reorders the top results.
| Method | How it matches | Best at | Weak at |
|---|---|---|---|
| Keyword search | Exact term overlap, BM25 | Names, codes, rare terms | Synonyms and paraphrase |
| Dense embeddings | Vector similarity | Meaning and paraphrase | Exact identifiers, negation |
| Hybrid search | Keyword and vector scores fused | Balanced general use | Extra tuning effort |
| Cross-encoder reranking | Joint reading of query and text | Final precision | Too slow for full corpus |
Choosing an embedding model
Public leaderboards give a starting point, but evaluation on the team’s own queries and documents is decisive, because domain vocabulary, language mix and document length strongly affect quality.
| Situation | Recommended choice | Reason |
|---|---|---|
| Mixed-language content | Multilingual model | Aligns languages in one space |
| Legal, medical or code corpus | Domain-tuned model | Captures specialist vocabulary |
| Very large corpus, tight budget | Fewer dimensions or quantized vectors | Cuts storage and latency |
| Long documents | Smaller chunks, not longer inputs | One vector dilutes many topics |
| Asymmetric search | Model with query and passage prefixes | Short queries differ from passages |
Some models support truncatable dimensions, so a team can store shorter vectors and accept a modest quality loss in exchange for lower cost.
Limitations and common mistakes
Embeddings capture topical similarity better than logical precision, and several recurring mistakes follow from that.
- Negation and numbers blur: “refund allowed” and “refund not allowed” may sit very close together.
- Model changes require re-embedding: switching models means recomputing every stored vector.
- Oversized chunks dilute meaning: one vector for a long section averages many topics.
- Mismatched preprocessing between indexing and querying silently lowers quality.
- Privacy is not guaranteed: embeddings can leak information about the source text and need the same protection.
The short version
Embeddings turn words, sentences or pictures into lists of numbers that work like map coordinates. Things that mean similar things get coordinates close to each other, even if they use different words. “Car” and “automobile” end up as near neighbors.
This lets a computer find information by meaning rather than by exact wording. When you ask a question, the system turns it into coordinates too and looks for the closest neighbors on the map, which are usually the most relevant passages.
Where it makes a difference
An insurance company has 40,000 internal policy documents, and claims staff struggle with keyword search because customers and documents describe the same situation in different words. The team splits the documents into passages of about 400 tokens, embeds them with a 1,024-dimension model and stores the vectors in an approximate nearest neighbor index.
Testing with 300 real staff questions shows that pure embeddings miss queries containing policy codes such as form numbers. The team switches to hybrid search, combining keyword and vector scores, and adds a reranker for the top 50 candidates.
In the evaluation set, the share of questions with a correct passage in the top five results rises substantially compared with keyword search alone, and staff spend less time searching manuals.