Embedding
In shortAn embedding is a list of numbers that represents the meaning of a text or image, so that similar items end up close together.
In plain words
A computer cannot compare meaning directly, but it can compare numbers. An embedding model reads a piece of text and returns a long list of numbers, a vector. Texts with similar meaning get vectors that lie close together; texts with different meaning lie far apart. Early work on word vectors, such as the 2013 word2vec paper, showed that such vectors capture word similarity. Today embedding models from OpenAI, Google and others do the same for sentences, pages, images and more.
To compare two items, you measure the distance or angle between their vectors. A small distance means a similar meaning.
Why it matters
- Search by meaning: A query like "I forgot my login" finds a page titled "Reset your password", although they share no words.
- Building block for RAG: Embeddings find the passages to hand to the model.
- Other uses: grouping similar texts, spotting duplicates, recommending related items.
- Limits: Embeddings can miss exact matches such as names, codes or numbers. Many systems mix them with keyword search.
- Practical rules: Use the same embedding model for your index and for your queries, because vectors from different models do not compare. If you change the model, rebuild the index. Embedding models are separate from chat models and are priced separately; see AI API prices.
- Privacy: The texts you embed are sent to the provider unless you run the model yourself.
Example
Three sentences: "How do I reset my password?", "I forgot my login", and "Best pizza in Rome". An embedding model turns each into a vector. The first two sit near each other. The third sits far away. A help-centre search uses this to show the password article for the second sentence.
Common misconceptions
- "An embedding is a chat answer." It is a tool for finding and comparing, not for writing.
- "A longer vector is always better." Longer vectors can capture more detail but cost more to store and search.
- "Embeddings are anonymous." They derive from your text. Treat them with the same care as the text.
Questions and answers
What is an embedding in simple terms?
A list of numbers that stands for the meaning of a text. Similar texts get similar numbers, so a computer can find related items.
What are embeddings used for?
Search by meaning, finding passages for RAG, grouping similar texts, spotting duplicates and recommendations.
Are embeddings the same as tokens?
No. Tokens are the small pieces a model reads. An embedding is a vector for the meaning of a whole text, or of one token inside a model.
Read next
- RAG (retrieval-augmented generation)
RAG (retrieval-augmented generation) first looks up relevant passages in your documents, then gives them to the model so it answers from them.
- Token
A token is a small chunk of text, often a word piece, that an AI model reads, writes and bills.
- Large language model (LLM)
A large language model (LLM) is an AI system trained on huge amounts of text to predict the next token, which lets it write, summarise and answer.
- Hallucination
A hallucination is a fluent, confident AI answer that is false, invented or not backed by the sources it was given.