Vector embeddings turn data like text into number arrays, and vector databases store and search those arrays as points in high-dimensional space to find nearest neighbors by meaning rather than exact keywords, a capability that traces back to the 2013 word2vec paper and now underpins retrieval-augmented generative AI systems.
What Is an Embedding?
An embedding is a numerical representation of a data point, expressing non-mathematical data such as words or images as an array of numbers that machine learning models can processCITE:E1. IBM (IBM) describes vector embeddings specifically as representations that let ML models handle data types — including text and images — that are not natively numericalCITE:E1.
How Do Vector Databases Store These Vectors?
Vector databases provide the ability to store and retrieve vectors as high-dimensional pointsCITE:E2. Amazon Web Services (AWS) frames this storage-and-retrieval function as the core capability that distinguishes a vector database from a conventional databaseCITE:E2.
How Do Vector Databases Find Similar Items?
Vector databases add capabilities for efficient and fast lookup of nearest neighbors in N-dimensional spaceCITE:E3. AWS states that this nearest-neighbor lookup is what makes similarity search possible once vectors are stored as points in that spaceCITE:E2CITE:E3.
How Does Vector Search Differ From Keyword Search?
Vector search finds what users mean, without requiring an exact keyword matchCITE:E4. Elastic contrasts this semantic matching directly with keyword search, which depends on literal term overlap between a query and a documentCITE:E4.
What Role Do Vector Databases Play in Generative AI?
Vector databases can provide an external knowledge base for generative AI chatbots and help ensure they provide trustworthy informationCITE:E5. AWS points to retrieval-augmented generation (RAG) chatbots as the specific use case where a vector database supplies external, retrievable context for a generative modelCITE:E5.
How Is Embedding Dimensionality Chosen?
Embedding vectors have a fixed dimensionality, and OpenAI's own models illustrate the trade-off: text-embedding-3-small outputs 1536 dimensions by default, while text-embedding-3-large outputs 3072 dimensions by defaultCITE:E6.
| Model | Default embedding dimensions |
|---|
| text-embedding-3-small | 1536 |
| text-embedding-3-large | 3072 |
OpenAI's documentation states that the length of the embedding vector is 1536 for text-embedding-3-small or 3072 for text-embedding-3-large by defaultCITE:E6. Higher dimensionality carries more granular semantic information but also increases storage and computation costCITE:E6.
Where Did Vector Embedding Technology Come From?
The technique of representing word meaning as vectors traces back to a January 16, 2013 paper, "Efficient Estimation of Word Representations in Vector Space," published by researchers Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean and posted to arXiv as the word2vec paperCITE:E7. That paper proposed representing words efficiently as continuous vectors, work now regarded as one of the foundations of modern embedding techniquesCITE:E7.
What This Means
Taken together, these facts describe a single pipeline: data is first converted into number arrays via embeddingCITE:E1, those arrays are stored as points in high-dimensional space by a vector databaseCITE:E2, nearest-neighbor lookup over that space enables similarity searchCITE:E3, and the resulting matches are semantic rather than literalCITE:E4. That same storage-and-retrieval capability is what AWS describes generative AI chatbots drawing on as an external knowledge baseCITE:E5, while the dimensionality choice behind the vectors themselves — 1536 versus 3072 in OpenAI's own models — sets a direct trade-off between semantic granularity and compute costCITE:E6. The underlying idea is not new: it traces to the 2013 word2vec paperCITE:E7.